{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.导论"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "什么是机器学习？机器学习的一个重要的目标就是利用数学模型来理解数据，发现数据中的规律，用作数据的分析和预测。数据通常由一组向量组成，这组向量中的每个向量都是一个样本，我们用$x_i$来表示一个样本，其中$i=1,2,3,...,N$,共N个样本，每个样本$x_i=(x_{i1},x_{i2},...,x_{ip},y_i)$共p+1个维度，前p个维度的每个维度我们称为一个特征，最后一个维度$y_i$我们称为因变量(响应变量)。特征用来描述影响因变量的因素，如：我们要探寻身高是否会影响体重的关系的时候，身高就是一个特征，体重就是一个因变量。通常在一个数据表dataframe里面，一行表示一个样本$x_i$，一列表示一个特征。      \n",
    "根据数据是否有因变量，机器学习的任务可分为：**有监督学习**和**无监督学习**。\n",
    "   - 有监督学习：给定某些特征去估计因变量，即因变量存在的时候，我们称这个机器学习任务为有监督学习。如：我们使用房间面积，房屋所在地区，环境等级等因素去预测某个地区的房价。          \n",
    "   - 无监督学习：给定某些特征但不给定因变量，建模的目的是学习数据本身的结构和关系。如：我们给定某电商用户的基本信息和消费记录，通过观察数据中的哪些类型的用户彼此间的行为和属性类似，形成一个客群。注意，我们本身并不知道哪个用户属于哪个客群，即没有给定因变量。      \n",
    "\n",
    "![jupyter](./1.1.png)        \n",
    "\n",
    "根据因变量的是否连续，有监督学习又分为**回归**和**分类**：\n",
    "   - 回归：因变量是连续型变量，如：房价，体重等。\n",
    "   - 分类：因变量是离散型变量，如：是否患癌症，西瓜是好瓜还是坏瓜等。\n",
    "\n",
    "为了更好地叙述后面的内容，我们对数据的形式作出如下约定：      \n",
    "第i个样本：$x_i=(x_{i1},x_{i2},...,x_{ip},y_i)^T,i=1,2,...,N$     \n",
    "因变量$y=(y_1,y_2,...,y_N)^T$        \n",
    "第k个特征:$x^{(k)}=(x_{1k},x_{2k},...,x_{Nk})^T$     \n",
    "特征矩阵$X=(x_1,x_2,...,x_N)^T$\n",
    "\n",
    "在学习机器学习中，我们经常使用scikit-learn简称sklearn工具库来探索机器学习项目，下面我们开始使用sklearn来演示这几个具体的概念："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:32:35.687512Z",
     "start_time": "2021-03-14T11:32:34.337510Z"
    }
   },
   "outputs": [],
   "source": [
    "# 引入相关科学计算包\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline \n",
    "plt.style.use(\"ggplot\")      \n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，我们先来看看有监督学习中回归的例子，我们使用sklearn内置数据集Boston房价数据集。sklearn中所有内置数据集都封装在datasets对象内：\n",
    "返回的对象有：\n",
    "   - data:特征X的矩阵(ndarray)\n",
    "   - target:因变量的向量(ndarray)\n",
    "   - feature_names:特征名称(ndarray)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:33:11.644406Z",
     "start_time": "2021-03-14T11:33:11.150401Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "      CRIM    ZN  INDUS  CHAS    NOX     RM   AGE     DIS  RAD    TAX  \\\n0  0.00632  18.0   2.31   0.0  0.538  6.575  65.2  4.0900  1.0  296.0   \n1  0.02731   0.0   7.07   0.0  0.469  6.421  78.9  4.9671  2.0  242.0   \n2  0.02729   0.0   7.07   0.0  0.469  7.185  61.1  4.9671  2.0  242.0   \n3  0.03237   0.0   2.18   0.0  0.458  6.998  45.8  6.0622  3.0  222.0   \n4  0.06905   0.0   2.18   0.0  0.458  7.147  54.2  6.0622  3.0  222.0   \n\n   PTRATIO       B  LSTAT  Price  \n0     15.3  396.90   4.98   24.0  \n1     17.8  396.90   9.14   21.6  \n2     17.8  392.83   4.03   34.7  \n3     18.7  394.63   2.94   33.4  \n4     18.7  396.90   5.33   36.2  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>CRIM</th>\n      <th>ZN</th>\n      <th>INDUS</th>\n      <th>CHAS</th>\n      <th>NOX</th>\n      <th>RM</th>\n      <th>AGE</th>\n      <th>DIS</th>\n      <th>RAD</th>\n      <th>TAX</th>\n      <th>PTRATIO</th>\n      <th>B</th>\n      <th>LSTAT</th>\n      <th>Price</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.00632</td>\n      <td>18.0</td>\n      <td>2.31</td>\n      <td>0.0</td>\n      <td>0.538</td>\n      <td>6.575</td>\n      <td>65.2</td>\n      <td>4.0900</td>\n      <td>1.0</td>\n      <td>296.0</td>\n      <td>15.3</td>\n      <td>396.90</td>\n      <td>4.98</td>\n      <td>24.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.02731</td>\n      <td>0.0</td>\n      <td>7.07</td>\n      <td>0.0</td>\n      <td>0.469</td>\n      <td>6.421</td>\n      <td>78.9</td>\n      <td>4.9671</td>\n      <td>2.0</td>\n      <td>242.0</td>\n      <td>17.8</td>\n      <td>396.90</td>\n      <td>9.14</td>\n      <td>21.6</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.02729</td>\n      <td>0.0</td>\n      <td>7.07</td>\n      <td>0.0</td>\n      <td>0.469</td>\n      <td>7.185</td>\n      <td>61.1</td>\n      <td>4.9671</td>\n      <td>2.0</td>\n      <td>242.0</td>\n      <td>17.8</td>\n      <td>392.83</td>\n      <td>4.03</td>\n      <td>34.7</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.03237</td>\n      <td>0.0</td>\n      <td>2.18</td>\n      <td>0.0</td>\n      <td>0.458</td>\n      <td>6.998</td>\n      <td>45.8</td>\n      <td>6.0622</td>\n      <td>3.0</td>\n      <td>222.0</td>\n      <td>18.7</td>\n      <td>394.63</td>\n      <td>2.94</td>\n      <td>33.4</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.06905</td>\n      <td>0.0</td>\n      <td>2.18</td>\n      <td>0.0</td>\n      <td>0.458</td>\n      <td>7.147</td>\n      <td>54.2</td>\n      <td>6.0622</td>\n      <td>3.0</td>\n      <td>222.0</td>\n      <td>18.7</td>\n      <td>396.90</td>\n      <td>5.33</td>\n      <td>36.2</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "boston = datasets.load_boston()     # 返回一个类似于字典的类\n",
    "X = boston.data\n",
    "y = boston.target\n",
    "features = boston.feature_names\n",
    "boston_data = pd.DataFrame(X,columns=features)\n",
    "boston_data[\"Price\"] = y\n",
    "boston_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:33:15.385401Z",
     "start_time": "2021-03-14T11:33:15.228402Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "e:\\a_projects\\pyprojects\\py_38_project\\venv\\lib\\site-packages\\seaborn\\_decorators.py:36: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEaCAYAAAD3+OukAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABslklEQVR4nO29eXyU1b34/z7PMzOZmezJJEAISxJAEAEFFcG6h7oAghZorXa1t7XS2tra6u29bW2vtra9LleLt/ptr1t/rXUrCrhUxB13BQQR2SEEyELIOpNZnvP742QmM1knMEkmmfN+vXiFPHmemXNmnud8PuezCimlRKPRaDQpiTHYA9BoNBrN4KGFgEaj0aQwWghoNBpNCqOFgEaj0aQwWghoNBpNCqOFgEaj0aQwWghoUoYHH3wQm8022MPQaJIKLQQ0Q5Kvf/3rCCEQQmCz2Rg3bhzXXHMNtbW13V7zxS9+kQMHDgzgKDtz8803I4Rg2bJlnf5ms9l48MEHY4599NFHLFmyhMLCQhwOB+PGjeP73/8+hw8fjpzzzW9+k0mTJtHc3Bxz7XXXXcfYsWM5evRof0xFM0zQQkAzZDnrrLM4ePAge/bs4e677+bJJ5/kq1/9aqfzpJQEAgFcLhcjRowYhJHG4nQ6eeKJJ3j77bd7PO/5559nzpw52Gw2Vq1axY4dO7jvvvtYv349p556Kvv37wfgf/7nfwiFQvzoRz+Kufbee+/l4YcfJicnpz+noxnqSI1mCPK1r31NXnDBBTHHbrnlFmkYhrz33nulaZpy3bp18uSTT5Z2u10+++yz8oEHHpCmacZc8/7778sLL7xQZmZmyvT0dHnaaafJt99+O/L3f/3rX3Lu3LnS6XTKoqIi+fWvf13W1NQc87h/+ctfyrKyMrlkyRJ55plnxvzNNE35wAMPSCmlbGlpkSNGjJAXX3xxp9eor6+XhYWFcuHChZFjb775pjRNU65atUpWV1fLkSNHyhtuuOGYx6lJHfROQDNscLlcWJZFMBjEsixuvPFG7rjjDj799FNOPfXUTudv2bKFs88+m9zcXNatW8dHH33E9ddfj2VZAKxbt45FixbxpS99iU2bNrFy5Ur27NnD5ZdfjuxDtZXw60Vz22238d577/Hkk092ec2//vUvDh8+zM9+9rNOf8vKyuJ73/sea9asiZh65s6dy0033cTVV1/NlVdeSWFhIbfcckvcY9SkMIMthTSaY6HjTmDLli2ytLRUzp49Wz7wwAMSkK+99lrMNR13AldddZWcPn26DIVCXb7HOeecI2+88caYY3v37pWA/Oijj6SUUvr9fvn73/9eTpkyRRYUFMgLL7xQPvLII7K+vl76/X756KOPyp/85CeR68M7ASml/OEPfygnTJgg/X6/lDJ2J/C73/1OAvLIkSNdju3JJ5+UgHz33XcjxwKBgBw3bpwUQshNmzb19PFpNBH0TkAzZHnllVfIyMjA5XJx0kknUVpayt/+9rfI30877bQer//ggw+44IILMIyuH4P33nuPu+66i4yMjMi/E088EYDt27cD8Pe//53169dz77338txzz3Huuefyhz/8gezsbNLT07nrrrv44he/2OXr//znP6empob//d//PZbpd+L555+noqKCtLQ0XnvttYS8pmb4o+PlNEOW2bNn89BDD2Gz2SgqKsLhcADw2muvYZomTqfzuF4/bFL6yle+0ulvI0eOBGDhwoUxzuhZs2Zx0003UVNTg9PpJCMjo9vXz8vL4z/+4z/49a9/zde+9rWYv02aNAmAzZs3c9ZZZ3W6dsuWLRiGwYQJEwCoqqri6quv5sYbb6SwsJCf/vSnzJs3L/I6Gk13aCGgGbK4XK7IIngszJo1i5deegnLsrrcDZx66qls2bKlx/fIzc3t8rjH44lrDN///vdZsWIFt956a8zxz3/+8xQUFPDb3/62kxBoaGjgj3/8I/Pnz4+8/7e+9S2Ki4u5+eabsdlsrF69mq985Su8+eabOjdC0yPaHKRJWX7605+yfft2rrzySt5//3127tzJ448/zltvvQXAr3/9a55++ml+9KMfsWHDBnbu3Mnzzz/P1VdfjdfrTcgY0tLS+M1vfsPdd98d40B2u9088MADvPTSS1xxxRW8++677N+/nxdeeIHzzz8fh8PBihUrALjvvvtYu3Ytf/3rX7Hb7QgheOCBB9i+fTu/+c1vEjJOzfBFCwFNyjJt2jReeeUVqqurOeecczj55JO5/fbbMU0TgPPOO49169axadMmzjrrLKZPn871119PZmYmdrs9YeP40pe+xIwZMzpFHM2fP5/169fj9/u55JJLKCsr49/+7d8444wzeP/99xkzZgzbt2/nxz/+Mb/73e+YMmVK5Nri4mJWrFjBf/3Xf/H+++8nbKya4YeQHe88jUaj0aQMeieg0Wg0KYwWAhqNRpPCaCGg0Wg0KYwWAhqNRpPCaCGg0Wg0KcyQzCKprKzst9f2eDzU1NT02+snM3ruqTf3VJ03pN7ci4qKujyudwIajUaTwmghoNFoNCmMFgIajUaTwmghoNFoNCmMFgIajUaTwgxYdNDy5ctxOp0YhoFpmtx22200NTVx5513Ul1dTUFBAddff32P9df7GwEY+/eTsWcPlseDt6CArgorCcBVXY1RU4P0eMAwEFVVPV4z2ESPOd65JfN8wiRyvDYgd9MmjH37sMaOpW76dIL9cP2xjHmofS+aocOAhoj+8pe/JCsrK/L7ypUrmTZtGosXL2blypWsXLmSq666aiCHFEEAWevXI158EbO5GdM0sS9cSMPcuTEPW/g8Vq2CUAhHbS1Mm4a/rg7T7+/ymsGm45i7mxtSxndekhD3vOIhGMTzyCNw663g82E6nXj+4z+o+cpX4hIENojr+mMZc0LnqdF0YFDNQe+99x7nnHMOAOeccw7vvffeoI3FVV0decgA9XPVKnW8m/NsPh/s3AmrV2OOHt3tNYNNvHMzKiriOi9ZiHde8WC++WZkAQfUz1tvJXfTpriuz920Ka7rj2XMiZynRtORAd0JhLsnzZs3j/Lycurr6yOdkXJycqivr+/yurVr17J27VoAbrvttri7NvUFY+9ehN2OYRgxbQnTWlpwR71f+DzsdkRTExgGWBZGMIi97bqO1ww20WOOptPcNmzAGcd5yUK884oHsWcPorUVhGg/2NqKsX8/nvPP730s+/fHdf2xjDmR8+yIzWbrl+dpKJDKc49mwITAf/3Xf5GXl0d9fT233HJLp+w1IQQi+gGKory8nPLy8sjv/ZHl53a7sQcCOO12fGFtzjQJuN20RL1f+DxCIWw2G4Zlgc1GyGYj5PN1ec1gEz3mCF2MszA/n9Y4zksW4p1XPIwYPx7S0to1eQCnE2vMmLjut4IxYxBxXH8sY07kPDuSalmz0aTa3Ac9YzgvLw+A7OxsTjvtNHbs2EF2djZ1dXUA1NXVxfgLBhpvQQEsXAhtXaUwTVi4UB3v5ryg0wllZbBgAaEDB7q9ZrCJd25WcXFc5yUL8c4rHkJnngn/8R8Q3gU6nfAf/0Hd9OlxXV83fXpc1x/LmBM5T42mIwPSWczn8yGlxOVy4fP5uOWWW1iyZAkff/wxmZmZEcdwU1NTXI7h/qodJIACr5eWFI0O8ng81NbUDKkolERFzXg8Ho7W1AxIdJABZO3ejVFRgVVcTENJCVYX50XTX9FBqaYNR5Nqc+9uJzAgQuDw4cP893//NwChUIjPfe5zXH755TQ2NnLnnXdSU1PTpxDRwS4gN1zD9VLtoYhmoObeMdInrNUPVqSP/s5TZ+6DKgQSzWAKgWR7iBNJeO7DVcj1xEAtCO7qaux33NHZvv+jH9EyCOadVFsIo0m1uXcnBIZkKenBpNtwvYkTB+UhTjQ6Jr1/MWpqYgUAQCikjg+D+0cz9NBlI/pIjw/xMEDHpPcvlsfT7uANY5rquEYzCGgh0EeG+0M83IXcYKMjfTTJhjYH9RFvQQH2hQs7+QSGy0NseTyYptnJZj1chNxgI4GGuXNxTZyYUj4XTfKihUAfGe4P8XAXcsmABOU/0p+pJgnQQuAYGM4P8XAXchqNJhbtE9DEkIrhoRpNKqN3App2hlgpaY1Gc/zonYAmwlArJa3RaI4fLQQGAYHKHM3YuhV3dTVd104dBKqqdHioRpNiaHPQAJPUGbmFhSoaSIeHajQpg94JDDDhjFxbczP2o0exNTcnjcllqJWS1mg0x4/eCQwwZk0NjspK2LsXpEQIgWPcOKxkqB0jhA4P1WhSDC0EBhjT6YTKSggXb5USKivV8SRgOOdAaDSazmhz0AATDARg/vz2frF2O8yfr44nAUnrtNZoNP2C3gkMMFZuLv6GBsxlyzB8Piynk9ChQ1i5uYM9NJ0noNGkIHonMMB4CwqgvJzQ7t0EKioI7d4N5eVJ4XzVeQIaTeqhdwIDTFLX5ukpTyAJhJRGo0k8Kb0TGCz7d9j52jxlCgDpyWJ/D+cJRKPzBDSaYU3K7gS6S9pi4cJBff/BtL9H8gR0GWmNJmVIWSHQXRtF45RTwOUatPcf1F7FQzBPQFc91WiOj5QVAt21UaSqCsaNG7T3H2z7+1DKE0jG3ZRGM9RIWZ9Ad72CKSwc1PfX9vf46XY3leTRTDoXQ5NMpKwQ6K7ht1VcnJDX7+1B1w3Hj58ed1NJSnj3Yr/jDswHHsB+xx1krV+vBYFm0EhZc1B3oZr54vgfx3jMFEkdKjpEsDwezCFW9TQpfUGalCZldwLQbv9umjKFljgX4Hi28vGaKY7l/TXtDMXd1FDcvWiGNym7EzgW4nVEJqvTd7gxFHdTQ3H3ohnepPROoK/Eq+Frp+/AMdR2U0Nx96IZ3uidQB+IV8P3FhSoxLMhmnSlY+/7j6G4e9EMb7QQ6APxbuWH8oOeyrH3AyX8hlIuhmb4o4VAH+iLhj9UH/SUjV7RZbQ1KYoWAn1gKGv48ZKqTu1uy2gPd+GnSXm0EOgjQ1XDj5eUjV7RZbQ1KYqODtLEkLLRK7qMtiZF0TsBTQypYPLqCl1GW5OqDKgQsCyLm266iby8PG666Saqqqq46667aGxspLS0lO9///vYbFouDTbD3eTVJUOwjLZGkwgG1Bz07LPPMnr06Mjvf/3rX5k/fz733HMP6enprFu3biCHk1LoypW9M9QSzzSaRDBgQqC2tpYPP/yQCy64AAApJVu2bOGMM84A4Nxzz+W9994bqOGkFLpypUaj6Y4Bs708+OCDXHXVVXi9XgAaGxtxu90qEgXIy8vjyJEjXV67du1a1q5dC8Btt92Gpx+ddTabrV9ffzAw9u9HvPgi2O3qH8CLL5J2yilYY8ZEzhuOc4+XVJ17qs4bUnvu0QyIEPjggw/Izs6mtLSULVu29Pn68vJyysvLI7/X9GPFRY/H06+vPxhk7NmD2dzc6Xhozx6aolppDse5x0uqzj1V5w2pN/eioqIujw+IENi2bRvvv/8+H330EX6/H6/Xy4MPPkhLSwuhtuzMI0eOkJeXNxDDSTlSNvZfo9H0yoD4BL785S/zpz/9iRUrVvDDH/6Qk046ieuuu46pU6fy9ttvA/DKK69w6qmnDsRwkoaBctambOy/RqPplUGNx7zyyiu56667ePTRRykpKeH8888fzOEMKANZqC1VY/81Gk3vDLgQmDp1KlOnTgVgxIgR/Pa3vx3oISQFA12oLSVj/zUaTa/oshGDxEC3GdR5AhqNpit0eu4gMZDO2lTuEaDRaHpG7wQGiYF01sbbFhP0jkGjSTX0TiAO+qPj1EA6a8OmJ5vPh/D5kE4nQaezc5lk3VhFo0k5tBDohUSZUroTJAPhrJUeD47aWti5E6RECIGjrIyWDqYn3VhFo0k9tBDohURE8Qy6Td4wYNo02LsXAgGw2dTvRgdroG6sotGkHFoI9EIi2i0Odt9eUVWFv64Oc9kyDJ8Py+kkdOAAoqoK8vPbTww3VtGZxRpNyqCFQC8kIorHqKkBhwNz9OiYRXigNGzL48H0+wnt2EFkFl3MQTdWGRj6w8ek0RwrWgj0gregAPtxLoyysBBHbi489hgEAph2O+aCBbQUFvZ4XTyLRTzneAsKsF92GebGjRheL5bLRWjGjM5z0I1V+p1BNw1qNB3QQqAXEhLFY1nw8ccQDKrfg0H1u2V1e0k8i0WfFpRgELF1KzQ1ITIyoC1ru6v56szi/iNsGrQ1N7dHamnnu2YQ0UIgDo53YRQ1Nfjz87Glp8eEaIoezEHx+BHi9TWEzwvabJCTow7qhWdQMGtqcFRWKid9OFJr3Dgs7XzXDBI6WawjUiY8WcryeMA0CTqdBHJyCObmYk6ciL2lpdv3iKesRLylJwa6RIWme0ynEyorQbbt1aSEykp1XKMZBPROIAoBmC++iP3xxxNqr43xKzgcyj/w9tv4t2/H3s17dOmQzsjA5nSSsXUrlseDbBMuvTmtdT+B5CEYCGCfPx9Wr1bhunY7zJ9PMBAY7KFpUhQtBKLor1DOaL9C2sGDWA89BOnp2I8e7dYm3MkhnZGBY/p0/Pffj9kmoLjssriieRLh3NYkBis3F39DQ2y47qFDWLm5gz00TYqihUAUkVDOUaOOOZSzt2gd0dCA7fBhrIqKHm3CHR3SNqcT//33xwgo26OPYl19NfLb3ybk8xHqxmmt+wkkD96CAuzl5YRWrSKkBbImCdBCIApZWIjIzMR86imE3Y4pBLbFi2keOTKu67uL1mmcO5fMtuPm6afD5s0YDgeWy9WjTTjaIZ2xdStmlDnHcfgw7N2LGD+ewMGDGAsX0jRlSrcLu476SQ60QNYkG9oxHI1lqQVaCMSuXbB9O+L//o+0zz6Ly0HcnTkpa/fuyHFZXQ2XXAI+H0YoFLdNOOxcBrD5fCq6xGbDcjp7rAqqST7CArlpyhRatADQDDJ6JxCFqKkBtxuamiArSzlPTRPj3XdxlZb26hfoNgqnokJt/QELEFu3IubPh9xcQjk5cdmEo+36wudT9X8WLCB04ED7++gwQ41G00e0EIjC8nggGMQyDHA41EG7HSstLa4FttsonOLiSBRP6MABzHPOQa5ZgzVlCqG6urhswjFmhH37EB9+SKiiArze9vfR0T5DAl02QpNMaCEQhbeggMylS2HrVvD7lalmwQKlqcexwHYXhdNQUkJm+LjXi980MW+5hUBWVrfO3K4ImxG8BQVktbbCrl3qD9q5OGTQZSM0yYYWAlFIILRoEZbDgfHuu1hpaYQOHYLy8rgW2J6cfh2PN8Sx8HenMca8Xl0dNrudkM+Hq7paa5VJzmBXlNVoOqKFQEcMg6OzZ5NVWIhRUYGYN4+GkpK4F9boKJwuF/GoB10A7ro6XHv3Io4cIVBSQkNJifIb0LPGKFE7j6zt25GrVmGEQhhaq0x6ElGaXKNJJFoIdMSyyHnnnchOQL78Mpnl5Z0Kt8VT3bOnRVwAWR9+iPP99+Hee6G1lbT8fPJvuIHaxYtxJrB2kCZ50NnbmmRDC4EoBGA+/TTi178Gvz9S8tm/dm1kYY3XptvTAu0tKCB7927S9u6FTz4BpxNaW6G2FuPuu8maMYOgz9erxqi1yqGHzt7WJBtaCEThqq5GPP64cgqDqu2yejXmsmWE2hbWeLXvbhfoujqytm/H9te/IiorYcsWWLAA+dZbUF8PXi9GRQXW5Mm9aox90Sp1REpyoJPFNMmGThZrQwBpBw9CaSnizDMhM1P9IRDAaG2NLKzxVuSMTu6KYJrY7HZYtQppsyEdDlU64rnnEKefjnA4wO3GKi5WmuHChe2v0YXGGM854bllrV+P/Y47MB94APsdd5C1fn1CKqRq+o5OFtMkE3onQPsiaTzxBGzciPD5EOecg7VpE7S2Yp1+emRhjVf77mrbb5s3D3ngAKK5mZDTiZGVhW3qVNi8GdGWPSy/9S0a2xzRvWmM8WqV2neg0Wi6QwsBYpuu2MeNw9q1C+OttxCXX05o6lSOzp4dWVjjtelGL9BmTQ32hgaCL7yAOXIkxtatGEVFWDk5qozE9OnIs8+GmTOxfD6ctbU05+fHVe8nnnO070Cj0XSHFgLELpKyuBgrIwPp8xGaO5ejs2YB4I6ypzfOnYszDptueIF2A8bDD7dnDM+fD2vWYDNN5N69cMklWI88Ao2NYLfjmDWL5vz8hM2vL74DA8javVv5JYqLIyGrGo1meKKFAJ0XyaDTCenpBMaOBXqIBopTi47RxL1e/IC5dCnG2LHIkSORmzYpAQCq/3CCG4zEHZFiWeQ//TTiyScRJSUgBJ7PfY7a8nJCXb+0RqMZ4mghQIfibM3N2L1e5JIlNPYhGqgnOmniXi+h3bux5s1TXcyamtRxIaCsDH+b8DleoiOCghMnwg03IKqqut29mJs3w6OPIkaNgr/9TTWnX72avFtvpebCC7UDU6MZhmghgDLbNM6dS45pIj/6CMswCL3xBplOJyGP57jt6T3WFLr6amxPPIFoakJmZBBcsiQhztqO+QyR9+wpm3jXLjWnVavadyO1tRiPPopr5kztRE4QOlxXk0xoIdCGs7qa0MqV2Nvq8ACwahW2b38b2YcMz+4e8HhrCiVqQTimHUxGhvoZbY4SAoTQTuQEIYCsd9/F3LgRw+vFcrmwz5hBw+mna0GgGRS0EGgjYre329sPhkKEfD6MODM8e8om7o7+6vh1LBFB8oQTEKefDi+8oDKYDQMmTCCYm6vLGiQId20tjvXrI43mTbsds7kZd1lZQoMBNJp40UKgjYjdHsDlwhw9GqO1laDTSUOc0UDdad/uE07Atm3bgJYPPpYaNVZxMYETT8T5058innoKKQTWqFEEv/AFXdYgQTgOHIgIACCSle44/3wtBDSDwoAIAb/fzy9/+UuCwSChUIgzzjiDZcuWUVVVxV133UVjYyOlpaV8//vfx2YbHLkUsdu/8QYOhwMefxyKipCVlWRedhnBkpJeX6M77dtx4ADWACdrHVONGiFomDmT4LhxuE4+GdHYSGDcuEjymiYBBAIQDKrWom3fiyVEwiPCNJp4GZAV126388tf/hKn00kwGOQXv/gFJ598MqtXr2b+/PmceeaZ3H///axbt47Pf/7zAzGkToTt887x45G33IKcMkWFijocONavx/boowRtth61+O60bwIBCIWw+XwInw/pdBJ0OvvVzn48/gbbtm0E2oSHLRgkt7wc3+zZtLQlsKUSiXbi+seOxVFUBBs3qp7WhoExY0bCIsI0mr4yIEJACIHT6QQgFAoRCoUQQrBlyxZ+8IMfAHDuuefy+OOPD5oQALVwykCAQNhBCpijR8NjjyFKSyEnp0ctPkb7djgwi4sxxo0jmJeHw+eDTZtUrSAhcJSV0dLPdvZeext0cU20Sctx+LBqaL9pE2lf/CK2qVNTqldBv3QBMwysuXMRXi+itRWZloacO1f5XzSaQaBPQqCxsZGPPvqIuro6Fi1axJEjR5BSkh+HLdOyLG688UYOHTrEhRdeyIgRI3C73RE7fF5eHkeOHOny2rVr17J27VoAbrvtNjz9uHgara0409PbtflgECElZGZitgkygLSWFtxdjWPhQoyZMxGvvYZYuxa5fz+2tWth2jSorIQjR5Tz+ZRTSM/MxDUQDlcpMV98McY0lLlwIaF581T0Txs2m42MlhaE3Y7w+6GiAmFZ0NKC2dyMfPFF0k45BWvMmP4f8wBjs9k63VfG/v2IF19U31c4YOA4PwNj715EY6Mq/OfzIZxOxOHDpDc14TrhhOOdRp/pat6pQirPPZq4hcAnn3zC7bffTmlpKdu2bWPRokUcOnSIZ555hptuuqnX6w3D4A9/+APNzc3893//N5WVlXEPsry8nPLy8sjvNR0qdiYCgSoNkXHoENbZZ2O99BLBcE+B8ePxmyaEQ0dNk4DbTUs343A3NmJ/+WX1i8+H/ehRxJNPwne+g9y/H8vpJHTgAKHdu2l2Ovs9ZtxdXa2S0qLNVI8/TmDMmJjdjMfjocXtxm6zkTZmDEJKpN8PGzYgq6qQBw7g27OHRpcrwSMcfDweT6f7KmPPHszm5k7nhvbsoekYPwO32429oQHq6toP9nI/9SddzTtVSLW5FxUVdXk8biHw4IMP8sMf/pBp06bxjW98A4AJEyawc+fOPg0kPT2dqVOn8tlnn9HS0kKobZt95MgR8vLy+vRaiSK87Xf85S+IPXvAbsdctgxRUoJ3+nRV6bMPDtaODmLpdCpfwP79BCoq1EHTRBYWDkjT8b6Ei/oKCkifPh1x112IvXvB4YBrr8V66y2orY3ZDQ13+qMLmG4qo0k24hYC1dXVTJs2LfZim41Qx8WlCxoaGjBNk/T0dPx+P5s2bWLRokVMnTqVt99+mzPPPJNXXnmFU089te8zSACu6mpsTzwBO3cq26zXi/zb37C++EXkSSf12cHaVS0ix/TpiDFjsAOWy0VoxgzlGOwhaihRTsnweGzNze2O6fT0LhczZ3U1/nfeIW3iRER2NjQ0IB9+GHHxxcgzziCY4CiWZM6e7Y8FWzeV0SQbcQuB4uJiNmzYwMknnxw59vHHHzM2jqiGuro6VqxYgWVZSCmZM2cOs2bNori4mLvuuotHH32UkpISzj///GOaxPFi1NQgmppARj2KgQCG13tMETydFo+MDJg7F+v11xGNjYiMDJg6FaOuLnZhdjojGrooKIhrlxDPIuorKCB99mxYsQL8foTDgWP5chq7mFd41xByuZDNzYiMDITLhZw8Gf+uXVi5uX36LHqiXxyvCaS/Fuz+ShDUaI6FuIXAV77yFX73u99xyimn4Pf7uf/++/nggw/4yU9+0uu148aN4/e//32n4yNGjOC3v/1t30bcD1geDzIjAxHlJMVux3K5jslk03HxsDmd+O+/X2mSOTnqpFWrcF52GWzdqhZmIXCMG4e/qAjL44mr7EO8JQic1dX4t2/Hce21iPp6ZHY2/u3bcVZXd4pwCu8agk4njtGjkXv3Im02Ql4vlJcn1GwxFJrd6AVbM9yJWwhMmjSJP/zhD7z++us4nU48Hg+/+c1v4ooMSna8BQXYlyzBUV8PbT4BFixoN9msXYtZUoLh8ymnblTj+Wi61MoLCsjYuhWzg9nM1tyM/PBDxPz57RmklZXYvvY1GgoKSN+6tVc7frwlCIy6Ohx2O6xYgQwEVM+CBQsI1dV1WtyidzH+ESOw5edDeTmt/ZAnoJvddE0ym8g0w4+4hUAgECArK4tFixZFjgWDQQKBAPboejtDkLDm7p44kYxDhwhKiX/0aFry80n/7DMcWVnw2GPtC20XC2hPpo2uHIwiGFTVSuvqMJctiwgYf3ExkvickvGWILDZ7bBmTex5a9Zgi/ouO34W4V1MoB8Xof5wvA51kt1Ephl+xJ2hcsstt7Br166YY7t27eLWW29N+KAGAwk0FxQQOu886qZPj7R37HYB7SD4ujVtVFd32RBeLllC6NAh1Vtgxw4CFRWqx0CbzT2uJvJtJQhi6KIpTcjng6Ki9pwAIaCoqL1a6jEQCanduhV3dfUxNa2Pa479RMfxx/iDBpGe7qPBJBHftyY5iXsnsG/fPiZOnBhzbMKECezduzfhg0omQj4f9qIilTkrZbcLaE+mDVlQ0MnB2FhQQKbT2W3kSTxOSf/YsdjKylRUU3hsXTSlCXk8+IuKsGVnx0QHhbrQuOPRRBOlrQ5WpExX4zeXLkXMnDno2nYymsj07mR4E7cQcLvd1NfXkxN2bAL19fWkpaX1x7iShngX0L6aNuJZAHtzSrYUFGCLoylN2M4fXLUKnM4eNe54nLUdz7E1NyP++leyR42ivo/F5gbD8dpttdcxY5AwqLb4ZDSRDQUHvubYiVsIzJ49m//5n//hG9/4BiNGjODw4cM89NBDzJkzpz/HN+jEu4D2FFPekyZ1PAtgvJp0XzTueDTR6HMi9YWkxLZ6NVmzZiW9htjdHF0ffUTg5ZcHVdtNxmSyZNydaBJH3ELgS1/6Eg8//DA/+9nPCAQCOBwOzj33XK644or+HN+gc0wLbV0dtrYOZa7qaoRhdKtJeQsKVJTPgQMQCOAfO5aWPmig8WrS8Z4XT2KZ5fFgZmTgyM/HEAJZVIT89FOstLQhoSF26aj3+bD27h10bTcZk8mScXeiSRxxCwGHw8G3vvUtrr76ahobG8nMzIyNqx/G9GWh9RYUkLV9O3LVKoxQCEcwiPm5zxGcPJnQrl3g9aqTQyHMmhqydu9uD/MMBrGVlWG7+uqEa6Dxhh3Gk1gWLi1h3HknVFUh0tIQ116L/8iRIaEhdqVty/POI/Tmm7EnDtJcki03IRl3J5rE0aMQqKqqorCwEIDDhw/H/M0bXsxQSV8aRdh+KkIh0iorVfP2Dz7AdvbZmPn5+EEJAtPEdDox3ngjNsxz505sTzyRUA20L449Z3U1/k2bMJcubc+L2LQJ55lnRsYTLi3hOPFEjIwMpGEgn3oK8/LLCdXXJ72G2JW2nZaZCeGif2G0tgsk5+5Ekzh6FAI33HADDz/8MADXXXddt+f94x//SOyohjBh+6mjqgo++ijSOETY7cg1azCXLiW0ezcsXEgwEMDp9caGdEqJaGpKqAbaF8eeUVMDTU2Eduwg1PF4B5+APzMTh8cT8QkYra2EhoiG2FHbdufnq3BVre12SbLtTjSJo0chEBYAoBf6eLE8HtKCQUR1tRIAoMwNfj9MmABTphC47DK8BQW4qquxXC5Mu71dEAiBzMhIqAbaF8dePPbf6HP8I0aoyKlgkOCCBTQM1VaUQmhtV5OSxJUsZlkW3//+9wnoPqi94i0ogPJypMOhKpK2laCQGzYghcA3eXLE8estKFClKRYsUOe1xfkHlyxJqAZqeTztCVlhujF1xJPA1fGcYHo6gauu6nN4aLIR1nabpkzpk3NeoxnKxOUYNgwDwzDw+/1DvkREfyMB3+zZpFVXY7cs5M6dyA0boLUV2WFxl0Dj6aeTNWIE9tmzlWCYMKHbBehYa8r0xbEXb/6C1po1muFB3NFBl1xyCXfddReXXXYZeXl5MZFB2jEcS0t+PraJEwmtXYs5dizGiBFYp59O/ezZMYu4r6CAzPXrkatW4W9bnG2XXYYbEB0W17BzNzoxzL5kSVxRRBJonDuXrFGjMCoqsMaOJeB2k7516zEv4D3ZiIdqAbShOu5UQH83/UfcQuD//u//ANi0aVOnvw0Hf0HkJtu7F7fbja+gAGfbTSc9HuXcrarq8QaMvlGDU6ZgHzUKKioIFhfTWFJCRocIHde8eQQ/+KC9Qml2Nua772J79FGCNltMFI+7uhrHX/4SKREhhMBRX4974kSaezEdCYgIm5DDgSM3F/vHH+PPz4+Mo7W4WPVPcLnIPI4SAUO2xICUQ3PcKcCQvaeGCL0KgdbWVp588klOOeUUSktLWbx4MQ6HYyDGNmBE32TCbsceCJA+ezb+TZugqQlHbS1Mm4a/rg7T7++2uUvkRnU4cBQUIDZswMrMxJw0ifzx4wlWVRFyOFSIaCiEePJJ0s4+G/n//h8EAtjOOguxcSOyqAiys2OieBz79rXXCAL1c+dOHPv29SoEoqODzNGj4bnnEFOmkObxgGUh//hHHBddhDh4kJyLLiJ0HCUhkqXEQF81R6OiIinGrelMstxTw5VehcBf/vIXdu7cySmnnMI777xDU1MT3/zmNwdibANGzE1mt2NrboYVK1RRsc2b1eK7dy/msmWEduzo8gaMWWjHj8dYtQoBGCNHwp//rEw9Y8ZgnnVWJFdAWBZiwwZV4x8Qfj/s2oUoLm4fXDiKx26H3FzE5MmIUAhpmshPP1XHe8GoqUGEQtgbGzGamhAnnIBcvVp1DPN6EQsWqAUyFML46CPMo0eRoRCG34+sqFACKs6SEMlQYuCYNMeqKgiFsPl8MZ3ekj3xLRVIhntqONNrdNCGDRv4z//8T6666ir+/d//nQ8++GAgxjWgdLzJhM8Hfj9G24JARgbitNOwuVw4zjqLtNmzSd+yhfSokrrRr2Gmp2PU1yMKChD/3/+HqKpCtLRg2O0Y69bhOPlkcLmQQiDz8yMlnqXNBjk5yC7CM/1jxmBcfDHiuefUjuW55zAuvhj/mDG9zk96PKRVVGC89ho4HIgnnsDwelX0UiAAzz6LKCgAlwsjNxfjs88wDxzA2LMHMzsb0tPbS0L0UtK4u0gkm9M5YGWIj6kcc0EBjtpajE2bENu2YWzahKO2VpkCNYNKX6LbNH0nLnNQbluNe4/HQ0tLS78PaqDpGBsvnU6Ew4HldCLy8jA9Hnj+ecTOnXD0KEybhty6FfeoUZESD9E1d4y8PJVAVVraXu+/tVUtwAcPYlRW4hg1Cqu8nNaXX8Z2xhmYI0eqfILrrkN+/DH4/TFRPO7qaqz33kOMHYsIBpE2G/K99+Cyy7qdV9gk4tq5Ey64APbtg927obgY6usRwSA4nar2z6FDMGIEcs0axNKliHvvhYYGOHQI48Yb8R8+HJf21VUkkmP2bPyPPILZ1KS08ssuI1hS0sn53Ve6M/kck+ZoGPC5z2HMnQvNzUrwGYY6rhlUdNmK/qVXIRAKhdi8eXPkd8uyYn4HOOmkkxI/sgEk5iZDxb07li8ntGkTZmkpPPwwYswYqKtTpqGKCuSiRfDWW5ESD5GaO089pbTr+fPVQm63q5u2rAzxySfI1lbYvx9RWIjvxBOxpaVhrF+PbKsdxMSJcMUVhIqLsXJz26ODamrw5+Zic7lizBWim4Ut2iRiHDgAn32G+Pa3kYWFsH8/UkpwOMBmgzaBZ7a2IqZPx3rhBYxLLoHGRoQQiOefx3H22fjjKAnRMRJJeDz4V6+GpiZ1gsOBY/36Lp3ffREEfe3k1qvmWFuLEQwi7r8ffD5wOjG+/W3E0aMwDFqoDmV0SHL/0qsQyM7O5n//938jv2dkZMT8LoTgj3/8Y/+MbgAJnnACjowMEILgyJE0FhTgPPNMzE8/JThzJmZLi1pMMzJUdI7NhjTNSIkHJ+Dfvx/neechDh5UvYpLS+Gaa+DNN+GTT5Tpp7wcPvoI4feTVl5Oa0kJtkcfRZSWRhZ2XnwR60c/ivE5RDeAx+lUB3tY2KJNIjIjA9PphN/8Bn78Yxg1CvHpp1gZGVhuN2LePOTpp0NuLtaTTyKOHMF66y3Mxkaor4cxYzAqKnDMnh1TSK4rYiKRQiHsTU04Jk3C73KB16sc0489higthZycY3by9eQsPBbNUbS0qB7Mra3qQFMTrFhB2uc+R1Pco9L0F7psRf/RqxBYsWLFQIxj0IjWKK1QCNLTsc2bF7nZZHY2wmYjlJGB4XYrU4FhQH09hsNBMDMTy+PBqKnBUVysFtoZM5BXXYV4/31oaUEuXQqffaZKR7zyilpYDQOzqgrDbifY5gsIY2tuxvz0U4CIxtPXha2TSeToUbUzaWyE999XwiktDfLyaN22jZBhYE6bhrV/P+Yzz2B4vchgEDFjhnqvc8/Fv2kTzurqHhfrjouztNkQL7yA49vfRu7fj3C7ITMTMWMG9lBImdx27eo0397ozuSTtm9fTIiuUVGBVVzcezmL2lq1MwoLAQCHA1lTE8doFEMplr1jSHQyjzXV6e/7Ku48geFKlxrl2rXk+nwEX3wRy+HANnYsxoYNyNpaRFkZTJumtHubDebPV3WAAOH1IkaPVk7WN96AzEy1gObnI+vrYe3aSEE5SksJHTiAWVaGjDJdOA4fhspKQlu3Yn/22RhTSUzCVy8LW7RJRDQ1YQWDiFGjoLgYWV4OW7YgHA5EejqO4mJaCgtBCI7Onk3uv/874q9/xairg+3b1e7lv/4L5yWXYNXVxWhjHW9Qs21xjo6yMXJy4N13sdavR1x4IcasWYT++U9EIIA9FIJ585C1tThXrsQ+YwYNp5/e603elcnHUVtL6MMPMSsqcOTmQlsuBKZJZm8mp7FjsbKzMRyOiJC1XC6s6EitDkTPXRYWYtu2bUjEsncVEp2sY011BiJHIuWFQFcapTlyJOKJJ5Tpx+vFD6TNmqVqAjU1IQIBGDmSYHY2rcXFqlREQQGZJSWICROUKWHVKmXjT0sDpxNr2TLEvn2I1lZkWhry/PMJ7dpFaM4czDYN39bcDJWVMH8+oQMHOpk4os0svS1s0TsHmZ6uzFcXXICsqcEIhRDPP6+09LQ0xJe/HLnOAo6cfz4ejwf55z/D6NHw1ltq9/LYYziiHNFd9uq9/HLM2tpIToNpWZCTg1VSgqytBY8HuWYNhteLCASUc/qttwgB5uuvYzY34y4ro7kXO3zHnZEtGIRp0whVVERMTgSD2NLTlQmtF5NT6KSTEMuXw913qzwOhwOWL6ehpKTL8zvNfeJEzLffVkIHejRzDfaOoWNI9FCIux/sz2ywGIgciZQXAhGN0uFQi0cwiC03l1BWVnsVUK+X0Pbt2EaPxnruOeVUFQKjrAzZtig6q6sJffopYvZsxE9/qpzDQqjom1Wr4IILCF5yCYbXq2r0HzgAfj9Wbi4tkyYpDf+zz+C88/Bv29befMbhIO3gQdL27UNs2RKTbNbTzRDtTLNqanAtW4bx1FOISZOU8zMvT4WkmibW2rU4LrkETjghcm2ooQFj925EU5MSZCNHIjMy1ELeRpc36NatiKlTkXv3tn8G06cTSksjlJOjNH8pERkZSgD4fNDaqnYNgQCsXo3j/PN7FQKdHNCZmfiffx68XgyfL1KVVbQ5eXuNDhIC/9ixOL7/fUR9PTI7G//Ysd0uNB3nbni9qhdEWOi0fR4d3zMZsl+HWty9ALLefRdz40b1/Lhcce8YhzoD8V2lvBDwFhRgv+wy1d3rsccQUiJLSjBOPhkzMxOjvh7L6VQRNePHKxNQIKB+TpsWCSE0amqwAgFkQwMi7PA0DLWwSUlISkJTp7Zn47bZ9H0FBWS++y7Gxo2INkds2qRJWHl5WGlpmI2NWA8+iGhqUrbzBQvaG9P0cjOEnWm2ggLSN21CSKkyjTMyYM4csNuVKWrz5tieBoA1erSaW12dGm9uLpx+OoZp4q6uxltQ0OUNahw9irV/P9ayZaoUhmnC889jjBhBCBAjRyIOHYKGBhWKW1urhMwXvoARCmEJ0WksXdHRAS2CQRxjx+J3uVSkk90OwSAyDic6tGUMr15NaORINW6/Hz7+GNeYMV0K2Y5zt5xOTJutXeh0857JkP061NpFumtr27vvBQKYdnvcO8ahzkB8VyknBLraVgajInTIzMRvmqStX48oK0O+9hp204Srrya4dy8yvLi1afOiqgry87E8HkL19dhPPFEtyjt2gBCIvDzk+PFYeXk0lpTg7BDmFn2Dy0AA0+dTzuesLIxAALFzJ4HiYkynU2nOq1e3Zy7HeTPkbtoEN9+sHMPZ2XD++fD3v2PY7cox++Uv4x8/HlfUNYGsLNLmzEHu3g3p6YjTTkO89Raypga724194UKCJ5yghFn0YuhyIYSIaUrjKCzEcqlXlzU18IUvKP9IKKR2JDNnqrwKy4KyMvxjx/Y6J1d1Naxd2153yemEzZsxp01D7NyJceGFsGULAaczvrjymhocWVnKjBReaBYsINTBBxKZZ4eHM3TgAOaCBch9+9QJ3bxnMmjhHUOikz3u3nHgQGz3vT7sGIc6A5EjkVJCIHorbmtuRgSDuJYswT9qVCRCx3Q6MY8exaqsRJx6Ksbo0cimJvjnPzGnTEFu2YI/XDU1ahH2FhTgPPdcuOsu5KxZSuu225GBAHLWLOQjj5BZXq62/VFfYPQNboRCaifxwguIm25CHj6MVVmJ6fEQdDpxjBsHe/di+HyE+nAzGG2+CAIBFR20ebPSVt1ucLuxtm9vN32FP6uqKnxeL47vfAcjJwf52GNYra0Ir1dp7qtWqZyGDjdoaMYMmDEjZuvuX7KEYFmZypx2uzHWrEGceKK6xuGA9HQ1jlmz8F9xRVwasVFX12nRZsECzEmTkD4fgYICzMmTcQiBf+rUXqODhGki16yJXWjWrMG2aFGX53d6OP1+/HPnErzyyh6T4JJBC482Faa1tBCIIzrIALJ2744JSrB6OD+hBALtSZdhgsG4doxDnYHIkUgpIRDeijsqKyMtEcX27ThvvRV/1IMpfD6w2ZQDt7JSmVBsNowJE2DLFtJMk1BODsHFiyOLsARlPtm3D1lTo8JCN29Wi+vOneByYXviCXIyMmgdNar9i4y+wcNhlZaFJSWhjIwYE4N/xAhs+flY555LcNQoMIy4ykHL4mJwuWDECOUIbNu9yPHjCebltSedtSEAm9OJzeMhZLcrwXT0qOp6FmXvduzbRzA3F8dXv4o8ehRr5MhItVSxdSs0NSEyMmDqVFry85H5+aRXV2P/9FNEWpoKnw0GEfv3w9e+RmDsWBrjbExjs9uhw6JtPPMM1rhxBGpqcIRCancVDGLOnElmL2W3ZSAARUWR+wIhoKiIkM/X9fn08HD2MaN6MLTwsKnQ7fHQ0ksYrAHkr1wJK1aA34/pcJC/fDm1ixcPiCDwjx2LraxM+cyCQbDZsCZNimvHqOmdlBICRk2NisAJP+gAfj9y40Zs8+YRfPFFAGRGhiqqtmOH0uhBafWZmXDiiZCVhczNVX6BKKTdrmrs+HwYO3bAiy8qwTB3Lo6GBoxdu5AjR2IcOoR9yRIa585F5uVhXHyxCiH96CNVqqG0lFAg0KWJIbh4MY3Tp/ep3LM3L4/0G25A3HKLivLxelUHs6ws5cSM0kTDuyW5ahWishKzshLxrW8h3W4oLIw4PcPhmM433miPaGpoIPvCCwmtWhWb+xBl8xZ1dSqJ7umnVYJWWw5CaNs25PPP9x7K2UbI58PecdEeMQJZU4Pj5JMxHngA6XIhW1qUc7s3u3thIf6iItUqM5yRnZ6OVViIu5uolGNJYOpOeADdvs9gk7V7d0QAAOrnihVkzZjB0W6ipxKJt6AA96WXIh96KPLdcOmlSWu+SiQ6RDTBWB6PsqvLqI/PbseSEn9xMdaPfkRaSws+txvb7t3Y9u1TzXNsNsSyZchHHoG6Oqzp0wnV1MCuXbhKSiILi3/0aGwLFigN1DSVtjtqFNJmw/j4Y/Xezc2YO3ZgPPggOaaJtXIl7N+PsW8f8uKLkYcOISdOJLRnTycTQ7ivQfYHH/QpUsgMhQg9/zzm1VerxXLmTOTDDyOKimLrExHruIz0D37vPcRPfkJo507sXi/CbofSUlVLKbwIt/kqjHffRTQ3t0fIQIzNO6zBW5aFYZrKMbxmDeLmm5UfJU4nacjj6bRoW3Y7ZlYWYsMGhJSIadPA4SBUWAiHD/dod7eKi2HhQoKrVilTmWnCZZf1S+x/R+GRDBFDPWFUVLQLgDB+vzo+AELAWV2Nf+NGzIsuavfHbdyIc+7cpA1pTRQ6RDTBeAsKcC1Zgti9G2GzIYRALl5MqLZWhWq2bY+ba2oQBQW4S0pwhaN2fD5VwrmkpNsQwJb8fGxz56oqolJiXHcd8q23EH6/KhN97rnwwguIxkYMu105WisrsQ4dQmRnI95+G/mf/0lLaWmnBjaioCCyUJi1tX2KFAr5fNidTkL/+pdasLOyEEuWEJo9m+DkyRGzkjF+fCTZK0zQ6VSmrNbWdhOPYSCnTlV+hrBADQQiD6jR0X4bVUXU8Hph4kSM9HR1vd8PlZUYO3d2+Zn29F3aOyzatnnzkP/6F2LqVFUg79FHwbIwi4pwLFumEuK6o4tG8wKw3XFHv0fyJEPEUE9YxcWYDkesIHA4ekykSyRGTQ00NcUEG0SOJ8Hn05/oENEEI4Gj559PvmUhXnoJCcjVq3F89audauJEwivdbmzPPos5dqzaFUydir2t4Fro0KEYh54EGk4/HVdJSSSLlIULSd+8WZWfeO89RHOzMiM1NiIsS8XTS6kcyIB8+23k+PE0T5kSMx5XdTVi1SrsR49itLRAXh7Wv/6FuXhxr5FCXWnNwdpagpMnY9u+PbIAifR07BddpJzOUTeeWVwM69djlJWp8Tud8NpriHPPVSasNid4+DMRS5YoU1gXVUQ56SSMceMQjz8OwaAKyf3855FeLzafT5lg4nCSdpVB3WpZOEpLMQ4dQuzerRzfUqoSHxs2wJVX9vqa0Rp6xtatA9JjIBkihnqioaSE/OXL201CvSTSJRrL48HMyMAMh+928ewNV3SIaD+QVl1N60svqeggnw9ZXEzwnXdwnnlmJ60rrKEFbTaCPh/O+fMR996LmZGBcLkwly/vVniIgoL2kgL5+ap0wuHDakdgtyMXLoTPPutsmkpL6/LhN2tqSNu5E7ZsActCSIk5cSJSiF4jhTppzRkZmBdeiLl/P3QwK4VeeKHdP9K2iBuTJmFWV8N99ylHrN2OsXQpVloaoqgI6fMhzzuP0KFDUF5O3dy5OKdNUz4YpxP/I49gq6lBtC2m4p131PvZ7cpn8vHHcOaZylwWp5O0Y54ApknavHkqCS0QgE8+AY9HhemOGNFjxdXukB6P6ioX3dKzrIyWBC8+yRAx1BMWULt4MVkzZgxKdJCvoID06dNjHNNdPXvDER0i2g+Eta6YapzdaF0xjWLy8rCeekqVlM7NxcrJ6VZ4dLTx4vUq2+lttyGPHFE1hV56CS65RNnBW1rUgrhgQbcajgOU0GgL5ZRCIPbtw5g1i0BblFK3kS9EOSPr6kirqCC4ciX2jmYlKaGpidY2/0jYLGKrqYmEYgLq52OPIVasINTSgrDbsSZMIDhzpooCol2jzty6Fefu3YjaWrXIjxyp3mf6dJUwZrcrU1tZGcGrruq90FsbXZlQQi++SNrixYiHHlJj3LdP9Wg4fBjzwgv7vqgahkoIDGc/d0gQTBTJEjEUTVf5NEdLSgbEB9ARZ3U1/nfewTZlSvuOrJtnb7gxbEJEa2pqWLFiBUePHkUIQXl5OZdccglNTU3ceeedVFdXU1BQwPXXX09GRkbC3z/6hrY5nciMjPb69tCt1hWtoRk+H3i9SJ8PK+wX6KYsQPbu3dj++lekzUbQ6VQLXWMj4p57lFbpdCJ/8hNCUiJ+/WvYtg3hcBD0+WDu3C4ffnn0qBIa4aQZux05fz7B1tY+PQh2n0+9hs2mMnaDQYynnyZt0SIVqulwRPwj4Xk5t22DUaPUompZyizm8ahy2Zs3YzmdBPftQ558cqeb03Q6MY4cQezbpxa46dNVxNWHH6qyzYYBEyYg6+oIZGTEfXN3WfPJ61WRSpWVsGiRWlQDAUR2NlZ5edyLavh+Sfv0U0LZ2fCVr0Qyx6MTBBNFX4sD9jdhJcb2xBOIpiZkRgb2XkJs+5NkN5f1N/1dRntAhIBpmnzlK1+htLQUr9fLTTfdxPTp03nllVeYNm0aixcvZuXKlaxcuZKrrroqoe/dUSuXYRt1WxP5nrSuaA3NcjqVc6yoqN0x3EF4RB6etugdIQSOceMQp56KuPtuFYfeljkr7roL43/+B6uuDvnhh6osREYGTJjQafyu6moMUGGaS5aoB7Otx7D1b//Wp8/AHDUK8eGHOMaNIzBiBLbCQlVRtLIStmzBcc01nbbZ1ujRWIAoLUW01f6hqgqysjBzclT009GjWF08lKK1FXHyybB1q7Inh0Jw2mnqs28TZsyZQzAjo0/mmo4mFAHYDx5UceSffqq09p/9DNncDBkZ+E87La4FLKbCZnMzxtatMH8+/ro6taPrBzNNV6ateENl+wN3dTWOv/wl1gxWX4974kSaB2HRHSizXKoyIEIgNzc30qLS5XIxevRojhw5wnvvvcfNN98MwDnnnMPNN9+ccCHQldnA/847iG9/m6DP1+P2KnorFqqrI23ChBhbeUfhEX4vq6REZbAGArB3L2LKFLVw+v1q8RNC1dSvqcFqbSUUHVO/di3Zo0YR9PliyhMHQyHSKiqgtBTrk09U3fs4nXPRn0G4xg1792J3u7GOHEFMmgSnngqTJ6ueAR222Q0lJeRfcw089FB7lu/y5YiHH0bs2qWS6i66CNPtBmJ3XvaGBuSWLXDddapYXHExbN+OOPFE1UvZZlOJeH5/nxbXjiYUu9er8g8efBC5fDli+3a45RaE3Y6cORM5ZQpizJheF9XozyrodOIoKoI1azCXLiW0e3fMd56oypbJFh3k2LdP3beZmSoizjSRe/fi2LdvUITAQJnlkpVh10+gqqqK3bt3M2HCBOrr6yPCIScnh/r6+i6vWbt2LWvXrgXgtttuw9OHxcLYu1fFtdvtMcelYWA76ywA3FKqmOeqKozWVjxFRZHm74AyfQBISdqcOUoLLizEKi4mP+q8yHvV1iIuvbQ9o9XhgBkzYMwYOOUU9drvvovweLAfOYKjtFQJh/x8pW3eeScyJ0fF4r/9topy8fmUPb25GeMXv4DiYkInnUReHA+CsXcvIitLZQxLibjySnj88Yhmy3nnIXfuxGhtxWlZpLW04I7+jKXEKClBXHSRSmZzOFSU0LhxkJeHME3Enj2kBQJ48vMxX3yxPeLoyBHE5z6nBEhNDfzoR8oclZPTHkrq88E99+CePBl39OfeGwsXYpxyClRVIQ4fRvzqV7B/Pxw4oHopn3ACFBbCiBE4X3yRtFNOwRozpsuXstlseDyezvfLuHEIjwdj+nRsX/lK+3cuZcw8w9p7aN682HsnDrq7Rzt9D/1AeN7RmJmZqoHSrl0x/S+MzMw+PXuJwti7V5Ur+fKXI60/OXyY9KYmXG2Vb4+FruaedEiJ+fLL8OGHynfodpM5cyah887r833WHQMqBHw+H7fffjtf//rXcbdpjWGEEMrW3AXl5eWUl5dHfq/pQ7cnt9uNPRDoFHkRcLtpqanpZC5ypqcTmDev+624y6UWP1DdqLp6L58PfD7ML3wBo7UVeckl2HNyEL/9rdLg09KQP/0pocZGjPp65L33KmHx+c8jtm8nMHEiQZ8Pu9eLuXcvNDWppudCwLhx+DIzaSwqgiNH4voM0jMycLtc8Le/qffJy0N87WtY48ZhffwxoYoK8HpxOp34AoHIZxOZV3U19pUrVYiew4EtEIDXXlOfw+uvq8WrvBxZXU3Lp59if/zx9iY5oRDmX/+qqpb+619w8CDC41EOcZ9PLTAnnUSovp7a2tq+azht30eOZWH3+zFdLiV8wmUuTJNAIECwuZnQnj00uVxdvozH46G2poZsy8JWUxPx56hJOAiUldHickW+c3d1NfbHH2+PMnM6CT7+OIFuqo72hNvtxm6zdQqB7Pg99Acej6fT85STlkba9OnK59NWDZfp02lNS+NoP4+nK9xuN/aGBlXRNkzUM3ysdDX3ZCO9thb3unUxvkCOHsU7enSfi+cVFRV1eXzA9lPBYJDbb7+ds846i9mzZwOqf3Fd2xdbV1dHVlZWwt/XW1CgipyZpjrQwYzT7Va8ujryGgL10Gds3Yq7upru5G/Me3m9hHbvJjBrFr6sLOT69cgTT0SedJL6+fzziOpqOHgw9r0PHMBoS8oRI0eqv4c1ZimhslL1C+4LlqXCMMOvU1eH/Ne/8JaVEZo6FVtjI/ajR1WNoq4qX7YVazMfewzx5JOIp55Suxoh1A6mrg5Wr1Zhqx2ceDJsTsjNRZaVIU84Aaqr1c3sdiutbscOzMOHcUd95n2lsaQE45vfRKalqQiWgwehvh750UfYPvsMR12dyrjuDilVuYxHHkFMmoSxdavq8taNz8isqcFRWYmxaRNi2zaMTZtwVFaq+fcRX0EBjunTMR9/HPHkk5iPP45j+nR8g+T0DLW0YB05glywABYuRC5YgHXkCKGWlkEZT2/P8HCm2wqqBw4k7D0GZCcgpeRPf/oTo0ePZsGCBZHjp556Kq+++iqLFy/m1Vdf5bTTTkv8e9NziFVvkQd9aWjR3Xulb91Ka14ejmAQUV2NtCxkfT2Gx4P17LPtDtfCQmR2NkZpKfa0NPWFz58P69erhdxuh/nzCfaxeqKoqcGfn48tPb1z0pPNpuzlXi8yK6tTPSRoK9b22muqnHQopD6v119X23PDUP9GjYKaGlwtLfhNU9n7QZVzaDNFyeJijOZm5Pz5iEceUT4Smw0WLFCF+Y7D8ZhWXY2vogLnTTfB3/6GvPJKxKOPYjgcWAcPIpYu7dGGbFRUtJfLCIUwly7FaG1FLFhAfReROqbTqaKQwnkexyqgSb4QyJDHgz87G1tTU/t4srMJDZLpZCDCJJOWAaigOiBCYNu2bbz22muMHTuWn/zkJwBcccUVLF68mDvvvJN169ZFQkT7g55CrHpL1OlrQ4uOyWLpW7eqsFTTpLWwEFtWlnqwMjKwTjxRLYp+v7qhP/kEc8kS5FNPqdo3Z58NVVUEr7gCEZ0p2eZHiZfwHCO5ES4XZnEx9qNHsTZvJlRRQcjrVQvYtm0x9ZAAQoEAaaedhnjySWXOcjjg4otVWerRo9WOwOMBnw/54IPYv/Md/K+8opyr6ek4vvc9QvX1GCedhBw1SkUKff7z7eWr33kH8bnPYR6HjdOoqcG2f7/yO2zcCHv3Ir/7XbXwO51YptlzaGdVVfs94PVGShSEfL4uF5tgIID9C19AVFQg/H6kw6ESD4/h4exL7spA0FVJjmPRvBPp0OzvMMlkJVxBNRwZhRBx99yIlwERApMnT+axxx7r8m+/+MUvBmII3dJlos5FF0Vu+JjtWGYmYto0xOHDuPbvjyRGdaRTWGpGhgpLfecd9aCnp8PChTRNnkxeVDq+mDgR+d57BKdPV4t+IIAxdSp8+imBtlaQx/IwxszR4VBN2N9+G/n++5g7d8Ymi3Wx+Jjhss1NTZESF+LFF+EHP1BNapxO5Lx5qgRGbS2hrCwCbclm0uNB7N5N2gsvwNGjKtdg3jy4916V/dw2J1FQ0F6m+hgIFweUhoEoKICpU1VYblqaynD+4Q9VGY/uKCzs1CCnp3BQ2dbAXrSNWbQJenkM+QPJljGcCM072YviDRVaCgqwXX11TM5GcMmShO4QUy5juCMdb3g5fjwNLlf7jRrejmVmYkyf3t4ApraWrK9/vcubupOfoakJ/6ZNXYalRqfjGzYbwX/+U72e06k05bo6jK9/nZDdfszaVEwTkYMHsR58kGB+vqqJE9WtjIqKLhcfo64OMjORdXUqWQyQ6ekqgunzn1cL7cqViJNPRjocWCNHRrS29Opq0u66S2nnlgUbNyIvu0zVF7rgAmXi2rcP2dCAcRxb3HBxQPnHPyKvvBLj9tuVcAoEYPx45FtvKbtyGx21VOuEEzo1yOlN4Bq1tYh//CNST8fopTZRT2NPtozh49W8ky3sdagybDKGk53oG97p8agWiG2Et2NixIj2HUFWFiGns9ubuks/Q1MTQZ+Ppg6F4SyIpOO7q6tVK8voa/1+WkeNOu4HJzxHo6ZG5QlAp25lHXdBkWttNiwpI74LKQQcPoxsalJ1gNo0epmW1il3wbVjR7sAAPVz717YuhUZ7tAmJbz4IjIqAuxY5ld3/vnkpKcjtm/HmjZNFahzuwnk5RFMS4sko3WlpZpLl3Jk7txO7T+7e9jSDhyAV19Vgiy8cL/6KmmXXNLnqI1ktHkfbyexRGf59nesfCqjhUAveAsKcF55Jbb330e43UrT7aFsBLRv72PCB6OqY3Z3Qw+ERtjR9BDdrcyYODF2F9SGf/RobBddhGzrg4zdjrFkidrVjB+PfcQIxHnn4Z81i4Zx42IWCwFgt6tSEZalkuJ8PlVSu6lJOaYXLEB6PASOsy6NBdTNnk12YSG2DRsQJSUqOsnphKiaTN1qqXEkk4UxWlsR06fDO+9EQvfE7NkYra3HNPZksnknopNYIk1cqWxa0k1lBplwOn/wjTcwpk3DGDECmZtLIGxb7uam9hUUkD57drut3+HA0Vb1sLdoo/7WCLsSNMHFi2mYPp38DrugMDF9EsJ9g2fMUH2DTzsNfw/jDBQV4Vy0CP7+d2U2OftsxD//iZw8WVX6DIXg5ZeRt9+esNLEgZwcHKedhrjvPqxQqFPVSaOmBhEKYW9sRDQ3I9PTIS0N10cfEXj55fgeNqcTsXGjSkoLJ1Q5nZDgjPfBIBGdxBKp0KSyaUk3lRlkYrpsbdiA47zzVAmBrCyCbc7drm5qZ3U1/k2bVJhhOKpn+3aydu9GtLRgf+MN5LPPdhlt1N8a4bEImo59EmKu6cX0EcjIIC03V0UTBYNQVga7dhEoK8MoKIgswr62+kTHQ1hrMrdsQT75JGRkYGRkEPR4YkIupcejSnC0mamEYWDMmEFgypT4H7amJmRNDWLs2IgQkG3NT4Y6iegklsiieKlcQM6oqQGHA3P06Pa15MAB3VRmoIi5+bxe/IC5dClMmUJg8uRuF89OnZBcLhy5ucg//AHj3HMRDz6IcDqxXK725I/zz++zLflYORZBc6zCSVRV4aurwzFpkio1MWaM8i+0tODPyVFltU0zITHoYaFtjBql8g+mTgXLwhg7VpUEaXtw7C0tqs7Q5s1qATdNGD8eMzs7pnNVTwuNlZWlIoq2b2/XdCdOVMeHOInoJJbIonip3FRGFhaqaL62Uu6m3Y65YEHPXfL6SGpUYDpGLI+nPUsRIlnAvsmTlUYZ53Xm6NGwZg3SZkOGK2k2NmK0CRijtRXbkSM9ZiMnCx2zpw16zqYOfxahts5jwZYWWLBAlfOGhPo9wkLbys7GmDEDsWYNPPMMxj/+oYRw24Nj7NuHtWEDcv58lRE7fz5s3Kgcx9GYJrKt0XzH+VkZGaowXVupcOl0ImfOxOqHUugDTUNJCSxfrvJBIFIwMJCT02vWfJh4MvHjJdkyqgeUjtn+waD63TrefXM7eifQA8dq1+x4ndHaGilBbQYCGKWlqjhXKKTCIgsLCe3ejf2FF5La4dWVkyo9qix3V3b0rrpCcf31+L74RVVFNYF+j4gz0rJg06b2BUjKmAcnoulGId1u5JQpqgx1+LvuodF8uLSCWLhQ5SfYbMhBLK2QSCzgyOLF5JxwghKYY8cSDASw/f73cTsnE2nCSbaM6oGku2z/vnbJ6wktBHrgWB21Ha8TTieyslJ1v9qzB/Pzn0e8/DIiKwvZ0KBaM4aLdSWxw6ujdmdrboYVK1SZ5R07uhx/lw/wli04S0qQNTVYTie+BAmBsPA1PvgAq7UVY+pUhGEQys0l0Fa6m4ICGktK8CxdCr/7HbS2ItLS4MYbqTv9dBxRfo+eGs2HPB4C2dnY6+qgza8RyMkZtNIKiUQAGevXwxNPIJuaMNLScJxwAv6oNqS93aeJjA5KtozqgaRTtj/oHsMDzbHawqOvE0BWeGfg9eKvq8O88UblD/jkE0IHDqiHC8DhIO3gwaSMh+6o3QmfTzkMfb52W3qHh7PTA5yXhzM7G665BguOKfywO8LCN3vUKNLefRf27CFkGHDoEPayMgJtD05adTWtmzZhnzMn4pg2PvsMR3V1l43mY2ibX8uUKZEIMNmWLGZfvpyGYbAodWwqYwQCsGcP5uWXE9q6VZ3UyyKcyOigZMuoHkh0j+FhQlc7ioa22kL2NWvab+42B7L14IOYNlvSxUN3fBil04lwOLCiyz10eDg75kwYp50Gt94KGRnqhj6G8MNeaSt9TEWFMgF1aEJi1NQgQyH8mZnKMQ04u1jUelp8nNXV+Ldvx3HttYj6emR2Nv7t23GGBckQxrFvX3utGlCf265dmHZ7u7DvZRFOZLhzMmZUDxQSaJ47l9yMjIhprm76dJ0xPBTpakfR8eY2i4vh7bcJhqOEksw81HG8wfR0HMuXE9q0SZ3QxcPZKWdi2jRlool2uPcx/LA7ott7Wq++ipgzByMzk0B+fkxv4Hg1y54Wn/TPPsNht6udQFuymGPBAkJ1dUPfRGG3K8HZVsbDMk2MnBxlNoMBX4STMaN6oDCB/H/8A/F//6d6lDid5H/zm9R88YuEer06PrQQGEQ63txGSwv+7dtjT0oi22dXD2NjQQHOM8/s9uEM50w4Fi9WIaKTJiEBIxTCCguCPoYfdkdMe0+fD/nmm0ghVGZvVMZ2b5pldEZ3cOJEuOEGRFVVzPxs4aJ60XXe16zBtmjRcc9jsPGPHo1twQKMp5+OdPKyli4lcPrphE46Ka5FONGZrsmUUT2Q5Hz6KeKhh6C5We3MmpsRDz1EzowZ1E6enJD30EJgkIm+ubusHZRkts+OD2NvoYJmTQ2OnTth3TqklMiPPsL81reQ//iHOqEt/DAR2cJh/0PowAHMBQsitZ5EMBizyHeVyBSaNQt55EinxSssIDouXiGfD3tRkaqDFC7xW1REyOc77nn0RKJq6EReZ+9e3G53zOt48/Nxn3yy2pnV1UFuLmRndyoJ0hOu6mpYuxazpKQ9tn/t2qTZ1Q4VbBUVqrzK/v3tmenjxmGrqAAtBIYfQ832GY+2Zzqd0NiImDMn0rQ89PHHcM89hOrrj6k4WXdEzDzhxL5lyzBaWwkuWBCTrdpVIpO5dCli5sy40/RDHg/+oiJs2dkx9aH6MzooUdp19OsIux17IBDzOq66Oozt2xH33x/ZCRjf/jauGTNojrOXRbgbXcckp2FhLhtIsrJU86LoAoyVlep4gtDJYklE2NwS+NGPCH3jGwR+9KOkcQp3RTwJQUHLwli8GMPvR3i9GH4/xpw5eNPTOXLWWRxNkACALtoQCgEnnECgQwJXT+PuKr7d1tyM89NPY5Kkwu8VbAsN7amMSKJIVAJWb6/j2rtX+TqampDBILKpCVasUMfjpFtzmd3ep7GmOoG0NOTll6te2qYJLhfy8ssJhP0zCUDvBAaZrrb3Q8X2GU9CkM1mQ+zejVi7NlJ3n6wsbOeck/DxhIWo+4QTcL7zDqxdS2D7dmwvv0xWlKbb07g7Oo0dhw9DZSWhrVuxP/tsjMbcNHcuObm5kaiNo1Om9KvATlQCVq8tVY8cUfkTUkZMXbK1VR2Pk8Eylw03WouKVC7K5z8f2UnLujpau2kafyxoITCIDPUSufFE2dhrauCxx5DNzeqA1wuPPYZ9wQIYNy7hY5KAtCyC77+POXky9i7s0T2NO9okZ2tuVlvv+fNVLkeUachXUEDe008jHnoI4fMhnE7yvvY1ahctStjOpiOJipfv7XUCJSU4Cwray38bBgiBVVRExtatcfkiBsNcNiwxDKwZMxAvvYRobUWmpSEvuKDHftl9RQuBQWSol8iNy4fR2KhaPmZlISwLaRhIKVV/4n7CqKvDUViI2LcP4fdjOByYxcURe3RP446OgDI//ZTQ1q2xyXxtGnNWUxPGn/+syn+0VSIVf/4zWdOnJy7fIXpOgKOlBceZZ8KePViWheVyEZoxIy4TVPSOU3o8cNll8M9/qj92+N4CeXk4v/lNuO22iE+AG29E7tyJ+eqrcSkriepTnOqIqir81dU4yssjmen+qqqe+2X3ES0EBpGhXiI3nvjtQEkJzgkTEJs2qU5fNhtMn37cDWR6wpaWpvIOnnlGRQfZ7YhLL8XWZkftatz2yZORtbWx83O7EaJD/FObxmzfsiUiAAD1c9cu7Hv3Hne+Q0ciTV4efljlWbz9NkZ2NoHiYlUptRe6i3gK3nADsqmJQIfoINfu3chHHoHLL1cFy2w2+POfsf/85yo2PQ5lJZVj+xOJ9HhwVFTAq68q5UkIHGVltOiyEcOD4ZAO31v8diAjA+e556ribGGfwIgRnZy1icSor4f334ezz263R7//PsayZd2O29222EcvmFYohKO2FtvUqVgNDUghCC5ejLegAHdmZmzHNMNACoFsy0BOJOEmL+LUU+Gll5AnnwyAOX48wTjCLrvbccqJE7FmzaKlQ/VUceRIbBlpKZVjt76+/dgQUlaGNIYB06Zh7NoFra0qZyMq+z0RaCEwiAy1kNBjwVFRoZyK+/aphisZGcicHBwVFf3WP0FKCTNnIlatiggeuXChOt4LnRZMy4I33sD40pcI1tcrrRjwjh9P5pIl8PDDkfcQX/0q3vHjEz6fcJMX4XAgzjlHxe6HQoiKChylpb2GXfbYmKQLAmVlOM87D/HXv0ZaZ3LllYTy8tpP6kVZGer+rmTBqKpS5U+uvBJaWsDthk8/Vce1OWjokwpbZtMwMN54o73ZvGEgGhowL764395TZmXBa68hg0GlMQWD6vd/+7der4020dl8Pti9WyW5HTpEqKICdu3CVVIChkHI48FYskRpaGlp/baDi5S+LiuDDz9EPP88BALI7GyMxYsjZq7u6HNjEsuCDz5Qn53Npn5+8AHMnav+HoeyMtT9XcmC6XZjOJ1w990RgWxceimm252w99BCYJAZ7unwxtGjsHVrJMIEIWDrVnW8nwj5fFhlZRhCRGzaVmlpXOGJ0SY64fMpU0hbQxz14qGIBu0/eBBz/PgY7TqRDrswDSUl5C9fDnV1iJdfVnNKS0O29WeONnN1PSkLtmxBuN3tvQ+2bOm2MYlRUYFsbIQpUyKfn6yvB6eT0De+EZeyMhBtEVMB0dSkFKhoYbpxozqeILQQ0PQr0mZT2mRra7t93ulUx/uJ4wlPjDbRhaukRkJEIcYMYvr97S1EO/wtkVhA7eLFeF5/HelygdutFmDTxLLZVAG7HjBqapT/4sQTI45y2dDQrTlIFBQgnE4ICwrDQEyYgDVqFE1xliqQhYU4Ro3qFKGVyLaIKYHPh+X1IkpL2/MEvF4VtZUgtBDQ9CtWfj5y/nzE00+32+fnz8fqx37KxxOeGG2is2pqsDc0EHrhBRUi2uF1BtKfYwHNkyfjnjq1vcyzEFBWhn/s2B6vNd1ujLy8SLRU2KTgME2MDz7oVDvIApgzB/bsUd+ZzQZz5vQ5/8E4ejQ2QuvLX+7rtFMe/9ix2MaNQ+7cGYkOiuc77wtaCGj6lVbTxDZtmrLPh52M06bRGl1KOsEcr68l2kRnAFmFhZFCc9E1iAban9NSUIDt6quxPfEEoqkJmZFBcMmSXm3soqlJCY5zz40ILPHZZ5hbtiA++qhT7SCamwlt2oRx1VURwW199BFcemncY3UcOIC1di1GVlbkPa21a3Fcckm/BQQMR471O+8LWgho+hUrKwtfVRWOc85pb76yYwdWAgpgJaqiZk+v37HQXGbUYtlf/pzu5nXMwi0YhNJSxAsvKLOcwwHnnYcMBFQtpA5OW//Ysdjy8uDFF9vzBCZN6pv22Waikrm5ETMGXm97LSFNXHRV8bYxShFJBFoIaPoVb0EB9jPPxJ9gs0lPIYj08Le+PDyDEeHSW2jlsQgemZWlko3aSncIvx9eeQVx8cUwciSmzRbjtPUWFOC+9FJkW0kM6XTCpZf26Tvzjx2Lo7AwEhUmDAMxY0ZCzRipQG+KSCLQQkDTr/RXGGxPCzSQkMV7MDK6+0PwdIyWkoaB8HgQr76KsXYt9rQ0zEWLIk5bZ3U1/o0bMS+6qD2yZ+NGnHPnxj8Gw8A6/XSMgwfbG9OcfnpCk5xSgYFQRLQQ0PQ7/WE26XGBbvt/l3/rwxgGI6O7PwRPx2gpYZqYmzfD0aPqnxAY776rEpLCY2hqio18Ch+Pcwyiqgp/XR3mFVf0ewjtcGYgFBEtljVDEsvjae8bEKZtge7pb32hU3+CAcjo7m3sAnBXV5OxdWukv4EB5OzeTd7rr5Oze3enh7pj7wMRCiHPPhtZXY0sLUWWlmJVVakG83GMIe55+P2IzZth82b10+8fUiVRkoFE3cs9oXcCmiFJbyU3EhG+ORgZ3T3Nq0t/wdKlavFesQL8fkyHg/zly6ldvDgS0tlpHq2tiF/9Co4cAdNUSWd2u/rXyxjixVdQQPrs2ZFxCYcDx/LlNOpEsT4xEKVlhIynoEqSUVlZ2W+v7fF4qOkmiWa4M9Tm3lN0UF8jh5Jp7t2N3V1djf2OO7A1N0cctsZ552H88Y8q6icciulyEbj//m5LWqfX1uL++98RL72EEQhg2e3ICy6g5YorIuGbxxt55a6uxv6//4sjPx8RLoFcW0vgu99NmrIRyfSd90SiouCKumlEo3cCmiHLcC250d28zJoaHJWVqqJkOHRzxgzE0aPIlpbIeUZmpio6140Q8Obn4z7hBITbDc3NiPR05JgxeKNs9cf72Zo1NTh27oR169pLII8bh6XLRvSZ/r7PB0QI3HvvvXz44YdkZ2dz++23A9DU1MSdd95JdXU1BQUFXH/99WT0Y3lhTeowXCtYmk4nxs6dEN33IC+vc+y934/owWbsqq1VJSGefjrSrYpFi3DNnJmwRC7T6VRd2cKGBimhslId1yQVA+IYPvfcc/nZz34Wc2zlypVMmzaNu+++m2nTprFy5cqBGIomBUhUQ/ZkQzQ0qB4J4Wbtdrvq0HbNNRCuJJqWBtdeS6vD0e3rOA4cUHkDZ5yhHMRnnAGvvqqOJ4hgIADz58eOdf58dVyTVAzITuDEE0+kqqoq5th7773HzTffDMA555zDzTffzFVXXTUQw9EMc4Z6x7ZuEQJr61bE/PntWbj79iGFgD/8AdHWOrL10097zsiWEmPKFFi9GtHW7U0sWNCutScAKzcXf0MDjssui/EJWLm5CXsPTWIYNJ9AfX09uW03RE5ODvXRXYs6sHbtWtauXQvAbbfdhqcfw8xsNlu/vn4yM1zmbowfj0hP7xTfL8ePx9nN/IbC3I1JkzAuuEDtctqKsnHllZCRAbffHonCcV17LfYTTsDdTWKW6fGoHg+hkMoRCIXgjTdwLF+euM8gLw/bySfDvfdG6g/1Nq6BZih85wNBUjiGhRCde7lGUV5eTnl5eeT3/vToD5WIgf5guMxduFxkzZsXG1Z30UU0uFzIbuY3FOYunE6yTj8dMy0Nw+vFcrlgyhSsp57CnDSpvWz2W28RmDOn2yiczIYGnCUlGFJihEJYpolVUoKvoYHGBH0G7upq7Bs2YF5+eXuy2IYNBLZt09FBg0TSRQdlZ2dTV1dHbm4udXV1ZCWgoJhGA8O3Y5sEGk4/HVdJSWReZk2N0uSj6cX0FfJ4CBQVYXe7EW3CJJCTE1e/hXhJRNaxZmAYNCFw6qmn8uqrr7J48WJeffVVTjvttMEaimYYkirho+mAs7Y20mNACIGjrIyWHhb0mESutp2SfflyGhL4WVkeD2ZGBubIke07gUOHdMZwEjIgQuCuu+7ik08+obGxkWuuuYZly5axePFi7rzzTtatWxcJEdVoNH3EMOCUUxBHjkTCPeUpp3Qq1GYCOVu3YuzbB6NG4a+uxly6FCMYJGSzEdq0CeeZZ3ZrqulrwpKvoID0U09FvPUWorUVIy0Nc84cnTGchAyIEPjhD3/Y5fFf/OIXA/H2Gk2f6O8+BYlE1NaqpLGpUyMOWGw2dbwt5t8EPH//O/zud9DairDbcf3wh3gbGrAfORLpvdydqeZY8i5ctbXw2WeIN94Anw/hdCLz83HNnq2byiQZSeEY1miShaGWaGaz2+HJJ5F+f/v4HA5sF14YOSdn69aIAACUsPj970m75x54/XV1rIeiZMdSzthx4ACsXo1lWUowWRasXo3j/PO1EEgykiNWS6NJEoZaolnI54OiItV7FtTPoqKIdg8oE1BYAIDKKwiF1G4Bei1K1mvZ7q4IBFRpi2jCLUY1SYXeCWg0UQy1RLOYSJ+2pKyOkT7W2LEYaWmxgiA9HTl5MmL8eMSIETG9kztyLH0V/GPHYisrizis+6NBuiYx6J2ARhPFQNRvTyS+ggLss2djbNuG2L4dY9s27LNn44sSWEenTIEbb0RkZqrs4MxMxA03EHrpJcTDD2Pecw+Z69fTXabOsfRVaCkowH/11VgzZyJPOAFr5kz8V1+dNDkCmnb0TkCjiSJcv932xBOIpiZkRgbBxYv7tZHM8eCqrlaVOs84AxEMIm02WLcO15ln0tw2ZgvwTZ2K85ZboLoaPB44fBjjlVeguRnDsnDU1+OeODFyTTTHkncxXHM1hiNaCGg0HbHZkFOmILxepMulom+SFMe+fbBpE1LK9gVWCBz79kUWdHdtLY5167BWr4ZAADMUgtmzEaedBq+8osw1O3fGXNORY2pwfwzXaAae5L27NZpBwFVdDf/8J6FQqD3TddcuXCUlyWnKsNuVkIp2uNps7dU7aY/UiZwTDMKqVYgbbuj2Gk3qoH0CGk0UxxQJM4j4R4+GBQtiSzYvWKCOh2mL1DFCIQy/H2kY4Ha3C4WurtGkDHonoNFEcSyRMINJS34+trlzMdPTI0XlQjNm0BIVi+8fOxZHURFs3Kji9Q0DTjoJq6wMY8kSQnZ7p2s0qYMWAhpNFAPR2DuRdFVUrpMD1jCw5s5FeL2R0hLWWWcRmDIFx5Qp+Nxu7bRNYbQQ0GiiGIpRLb05YEVVFf6mJhxf/SrU1yOzs/Fv20aopQXbWWfRkqSmLs3AoIWARtOB4RbVIgsLcTidsGIFMhAAux3HggW0FBYO9tA0SYB2DGs0wx3Lgo8/bi/jEAyq3y1rcMelSQr0TkCjGeaImhoCxcU4ZsyIJMD5a2sR2gykQQsBjWbYIwsLSXO54M9/jvQmTvvylwloc5AGbQ7SaIY99oYGePPN2Mqob76pjmtSHr0T0GiGOUZFBVZLC6K0FBEKIU0T2dKCUVEBuq1ryqOFgEYzzLGKizENA9nQENN4xiou1qYAjb4HNJrhTkNJCSxfrjp8gfq5fLk6rkl59E5AoxnmWEDt4sVkzZihTEPFxTSUlKADRDWghYBGkxJYwNGSEtDav6YD2hyk0Wg0KYwWAhqNRpPCaCGg0Wg0KYwWAhqNRpPCaCGg0Wg0KYyQUiZzqXSNRqPR9CN6J9CBm266abCHMGjouaceqTpvSO25R6OFgEaj0aQwWghoNBpNCqOFQAfKy8sHewiDhp576pGq84bUnns02jGs0Wg0KYzeCWg0Gk0Ko4WARqPRpDApW0V0w4YNPPDAA1iWxQUXXMDixYu7PO/tt9/mjjvu4Le//S1lZWUDO8h+ore5v/LKKzzyyCPk5eUBcNFFF3HBBRcMwkgTSzzf+fr163n88ccRQjBu3Dh+8IMfDPxA+4He5v7ggw+yZcsWAPx+P/X19Tz44IMDP9B+oLe519TUsGLFCpqbm7Esiy9/+cvMnDlzcAY7GMgUJBQKye9973vy0KFDMhAIyBtuuEHu37+/03ktLS3yF7/4hfzZz34md+zYMQgjTTzxzP3ll1+Wf/7znwdphP1DPPOurKyUP/nJT2RjY6OUUsqjR48OxlATTrz3e5hnn31WrlixYgBH2H/EM/c//elP8oUXXpBSSrl//3557bXXDsZQB42UNAft2LGDkSNHMmLECGw2G3PnzuW9997rdN4//vEPFi1ahN1uH4RR9g/xzn24Ec+8X3rpJS688EIyMjIAyM7OHoyhJpy+fudvvvkmn/vc5wZwhP1HPHMXQtDS0gJAS0sLubm5gzHUQSMlzUFHjhwhPz8/8nt+fj7bt2+POWfXrl3U1NQwc+ZMnnnmmYEeYr8Rz9wB3nnnHbZu3cqoUaP42te+hsfjGchhJpx45l1ZWQnAz3/+cyzLYunSpZx88skDOcx+Id7vHKC6upqqqipOOumkgRpevxLP3JcuXcott9zC888/T2trKz//+c8HepiDSkruBHrDsiwefvhhvvrVrw72UAaFWbNmsWLFCv77v/+b6dOns2LFisEe0oBgWRYHDx7kl7/8JT/4wQ+47777aG5uHuxhDShvvvkmZ5xxBoaROkvDm2++ybnnnsuf/vQn/v3f/5177rkHy0qd5pup801HkZeXR21tbeT32traiBMUwOfzsX//fn71q1+xfPlytm/fzu9//3t27tw5GMNNKL3NHSAzMzNiArvgggvYtWvXgI6xP4hn3nl5eZx66qnYbDYKCwsZNWoUBw8eHOihJpx45h5m/fr1nHnmmQM1tH4nnrmvW7eOOXPmADBp0iQCgQCNjY0DOs7BJCWFQFlZGQcPHqSqqopgMMj69es59dRTI393u9385S9/YcWKFaxYsYKJEyfy05/+dFhEB/U2d4C6urrI/99//32Ki4sHepgJJ555n3766ZEImYaGBg4ePMiIESMGY7gJJZ65Axw4cIDm5mYmTZo0CKPsH+KZu8fjYfPmzQBUVFQQCATIysoajOEOCinpEzBNk29+85vceuutWJbFeeedx5gxY/jHP/5BWVlZlw/IcCGeuT/33HO8//77mKZJRkYG11577WAP+7iJZ94zZsxg48aNXH/99RiGwVVXXUVmZuZgD/24ifd+f/PNN5k7dy5CiEEeceKIZ+5f/epXue+++1izZg0A11577bD6DHpDl43QaDSaFCYlzUEajUajUWghoNFoNCmMFgIajUaTwmghoNFoNCmMFgIajUaTwmghoNFoNCmMFgIaTTcsX76cb33rW/h8vsixl156iZtvvhkAKSXPPPMM1113HVdeeSXf/e53+dvf/kYgEADgueee48c//jHBYDBy/Zo1a/jpT39KKBQa0LloNN2hhYBG0wOWZfHss892+bcHHniAtWvX8r3vfY+HH36Yn/3sZ3z88cfceeedAFx44YW43W6eeuopAA4fPsxjjz3GNddcg2maAzYHjaYntBDQaHrg0ksvZdWqVZ0KyR08eJAXXniB6667jkmTJmGaJmPGjOHHP/4xGzZsYPPmzRiGwXe/+13WrFnDvn37uO+++7jwwgspLS0dpNloNJ3RQkCj6YHS0lKmTp3KqlWrYo5//PHH5OfnM2HChJjjHo+HiRMnsmnTJgCKiopYvHgxv/rVr6itrWXJkiUDNnaNJh60ENBoemHZsmU899xzNDQ0RI41NjZ223wkNzc35twpU6bQ2NjIGWecgcPh6PfxajR9QQsBjaYXxo4dy6xZs1i5cmXkWGZmZky11Wjq6uoiVSiDwSD3338/F110Ec8//zyHDx8eiCFrNHGjhYBGEwfLli3jpZde4siRIwCcdNJJ1NbWsmPHjpjzampq2L59e6Qz1xNPPEF2djbf+MY3mDdvHvfff/+Aj12j6QktBDSaOBg5ciRz5szhueeeA5Stf968edx999189tlnWJbF/v37uf3225k2bRrTp09nz549PPfcc3znO99BCMGyZcuorq7m5ZdfHuTZaDTtpGQ/AY3mWFiyZAmvv/565PdvfvObPPPMM9xzzz0cOXKErKwszjzzTJYtW4ZlWfzpT3/i8ssvZ+TIkQA4HA6+853vcMcdd3DKKaeQk5MzSDPRaNrR/QQ0Go0mhdHmII1Go0lhtBDQaDSaFEYLAY1Go0lhtBDQaDSaFEYLAY1Go0lhtBDQaDSaFEYLAY1Go0lhtBDQaDSaFOb/B3d4YRt/MtVTAAAAAElFTkSuQmCC\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.scatterplot(boston_data['NOX'],boston_data['Price'],color=\"r\",alpha=0.6)\n",
    "plt.title(\"Price~NOX\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以看到，数据给定任务所需要的因变量，因变量为波士顿房价Price是一个连续型变量，所以这是一个回归的例子。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "各个特征的相关解释：\n",
    "   - CRIM：各城镇的人均犯罪率\n",
    "   - ZN：规划地段超过25,000平方英尺的住宅用地比例\n",
    "   - INDUS：城镇非零售商业用地比例\n",
    "   - CHAS：是否在查尔斯河边(=1是)\n",
    "   - NOX：一氧化氮浓度(/千万分之一)\n",
    "   - RM：每个住宅的平均房间数\n",
    "   - AGE：1940年以前建造的自住房屋的比例\n",
    "   - DIS：到波士顿五个就业中心的加权距离\n",
    "   - RAD：放射状公路的可达性指数\n",
    "   - TAX：全部价值的房产税率(每1万美元)\n",
    "   - PTRATIO：按城镇分配的学生与教师比例\n",
    "   - B：1000(Bk - 0.63)^2其中Bk是每个城镇的黑人比例\n",
    "   - LSTAT：较低地位人口\n",
    "   - Price：房价"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.2 分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们来看看一个分类的例子，我们来看看大名鼎鼎的iris数据集："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:33:18.639401Z",
     "start_time": "2021-03-14T11:33:18.624401Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)  \\\n0                5.1               3.5                1.4               0.2   \n1                4.9               3.0                1.4               0.2   \n2                4.7               3.2                1.3               0.2   \n3                4.6               3.1                1.5               0.2   \n4                5.0               3.6                1.4               0.2   \n\n   target  \n0       0  \n1       0  \n2       0  \n3       0  \n4       0  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>sepal length (cm)</th>\n      <th>sepal width (cm)</th>\n      <th>petal length (cm)</th>\n      <th>petal width (cm)</th>\n      <th>target</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>5.1</td>\n      <td>3.5</td>\n      <td>1.4</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>4.9</td>\n      <td>3.0</td>\n      <td>1.4</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>4.7</td>\n      <td>3.2</td>\n      <td>1.3</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4.6</td>\n      <td>3.1</td>\n      <td>1.5</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>5.0</td>\n      <td>3.6</td>\n      <td>1.4</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()\n",
    "X = iris.data\n",
    "y = iris.target\n",
    "features = iris.feature_names\n",
    "iris_data = pd.DataFrame(X,columns=features)\n",
    "iris_data['target'] = y\n",
    "iris_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:34:18.931215Z",
     "start_time": "2021-03-14T11:34:18.667213Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEMCAYAAADAqxFbAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA6rklEQVR4nO3de3gTdboH8O/k1qQXeguVUq4tLQIKaJE7rEIVF0V0jyCCLCqsHquwrCsP6D4r+hwFFPAgUhYvKF7YFT3ugnBwq1WxR1ZYCygWkJabCAVKm5aWNmmSmTl/xIRMmza/tJPJTPp+nsdHkpnMvJm2eTPz+73zcqIoiiCEEEJ+oYt0AIQQQtSFEgMhhBAJSgyEEEIkKDEQQgiRoMRACCFEghIDIYQQCYOSOxMEAUuWLEFKSgqWLFkiWbZr1y68++67SElJAQDceuutmDhxopLhEUIIgcKJYefOncjIyIDdbg+4fPTo0Zg7d66SIRFCCGlGsUtJ1dXV2L9/P50FEEKIyil2xrBp0ybcd999rZ4tAMDevXtx5MgRpKenY86cObBarUG3W1FRIWeYIbFaraiqqorY/kOhlVgpTnlpJU5AO7FGQ5zdu3dv87WKJIZ9+/YhMTERmZmZOHToUMB1cnNzMWbMGBiNRnz22WcoKCjA0qVLW6xXVFSEoqIiAMCKFSuYkke4GAyGiO4/FFqJleKUl1biBLQTa2eIk1PiXkl//etfUVxcDL1eD6fTCbvdjuHDh2PBggUB1xcEAQ888ADefvvtoNumMwY2WomV4pSXVuIEtBNrNMSpijOGmTNnYubMmQCAQ4cOYfv27S2SQk1NDZKTkwEAJSUl6NGjhxKhEUIIaUbRWUnNbdmyBVlZWRg2bBg++eQTlJSUQK/XIz4+Hvn5+ZEMjRBC2iSKIhwOBwRBAMdxkQ6nhQsXLsDpdMJsNoccnyKXksKJLiWx0UqsFKe8tBInoJ1YvXHa7XYYjUYYDBH9ft0qg8EAh8MBl8sFi8UiWaaKS0mEhJvw8V8hVp5r8TyXlg7dHTMjEBGJdoIgqDYpeBkMBjQ1NYX+ujDEQojixMpzwPmzLZ+PQCykc1Dj5aNA2hMn3SuJEEKIBCUGQgjRqC+//BLjxo3DmDFjsG7dOtm2S4mBEEI0iOd5/OlPf8J7772HL7/8Elu3bkVZWZks26YxBkIICbNwTI44cOAA+vTpg969ewMApk6disLCQuTk5HQoVoASA4kSXFp6wIFmLi1d8VgIaS4ckyPOnz8vmXaanp6OAwcOdGCLV1BiIFGBpqQSIh8aYyCEEA3q1q2bpMD33Llz6NatmyzbpsRACCEaNHToUJw8eRKnT5+G0+nEtm3bcMstt8iybbqURAghGmQwGPDcc89h5syZEAQB99xzD/r37y/PtmXZCiGEkFaFa3LExIkTw9IVkxIDIYSEmdYmR9AYAyGEEAlKDIQQQiQoMRBCCJGgxEAIIUSCBp9JxPnfR6bWHAPe4WksQk12CIkMSgwk4vzvI8MbDIDb7Xk+kkERogGPP/44ioqKYLVa8cUXX8i2XbqURAghChBFsc3H7TF9+nRs3ry5w9tpjhIDIYSEWWF5Lbb9aPMlA1EUse1HGwrLazu03ZEjRyIpKanjATZDiYEQQsJIFEXY3TyKT9X7ksO2H20oPlUPu5uX5cxBbjTGQAghYcRxHKZenQIAKD5Vj+JT9QCA8X0SMPXqFHAcF8nwAqLEQCLO/z4yenMM3H6zkgiJBt7k4E0KAFSbFABKDEQF/KekJlmtqKqqimA0hMjPe/nI37YfbapNDpQYSJvC0auWkM7Ef0zBe/nI+xjo2JlDfn4+vvnmG9hsNuTm5uKJJ57Avffe2+GYKTGQNoWjVy0hnQnHcbAY9JIxBe+Yg8Wg79AZw/r16+UKU4ISAyGEhNmk7CSIouhLAt7koMbLSABNVyWEEEU0TwJqTQoAJQZCCCHNUGIghBAiQWMMpE3h6lVLCFEvRRODIAhYsmQJUlJSsGTJEskyl8uFdevW4cSJE0hISMDChQuRlpamZHgkAJqSSkjno2hi2LlzJzIyMmC321ss++KLLxAXF4dXXnkFu3fvxubNm/GHP/xByfBIFKN6DBJtzp49i9///veoqqoCx3GYNWsW5s2bJ8u2FRtjqK6uxv79+zFx4sSAy0tKSnDjjTcC8NwxsLS0VJU3lyLa5KvHaPZfoGRBiBYYDAYsXboUu3btwvbt27Fp0yaUlZXJs21ZtsJg06ZNuO+++wKeLQCAzWZDamoqAECv1yM2Nhb19fXo0qWLUiESQkjY2Bt5nCxzoskhIMasQ98cEyyx+nZv76qrrsJVV10FAIiPj0d2djbOnz+PnJycDseqSGLYt28fEhMTkZmZiUOHDnVoW0VFRSgqKgIArFixAlarVY4Q28VgMER0/6HQSqzhirPWHOPpDteM3hyDpHbsr7Mfz3DQSqzeOC9cuABDgN+pQOyNPA7stcPt8hS51de5UVvD44YxCR1KDl6nT5/GoUOHcMMNN0hiMhgMiImJCfm4KpIYjh49ipKSEhw4cABOpxN2ux1r167FggULfOukpKSguroaqamp4HkejY2NSEhIaLGtvLw85OXl+R5H8oZrVg3d8E0rsYYrTt7R5GsZ6s/taGrX/jr78QwHrcTqjbOpqQl6PduHevlhO1xOARzH+S6Ru5wCyg83YuBQS4fiaWhowIMPPohnnnkGFosF7l9+zw0GA9xuN5qaWv6Od+/evc1tKpIYZs6ciZkzPQN8hw4dwvbt2yVJAQByc3Oxa9cu5OTkYM+ePRg0aJCqKwMJIYRVk0MIWPnc5BA6tF2Xy4Xf/e53uOuuuzB58uQObctfRAvctmzZgpKSEgDAhAkTcPnyZcyfPx87duzArFmzIhkaiTJcWjrQLaPFf1SPQZQQY9YF7PkcY27/R7AoivjjH/+Ifv364eGHH+5oiBKcqPGpPxUVFRHbt1ZOfQHtxEpxyksrcQLaidUbZ2NjI2JjY5leY2/kcWBPI1y/jDGIogijkcN1I2PbPcbw73//G3fddRcGDBjgOxtZsmSJb+an91JSoDhVcSmJdF78yieB6ostF6R2hX7RcuUDIiQCLLF6XDcyVtZZScOHD8fZsy1viS8HSgwkvKovAvWXIh0FIRFnidV3eKBZKXQTPUIIIRKUGAghpB20MjzbnjgpMRBCSDvodDpfzYBaud1u6HShf8zTGAMhhLSD2WyGw+FAU1OTKmuuYmJi4HK5YDabQ34tJQYSXqldQ3ueEI3gOA4Wi3oHkzsy/ZcSAwkrmpJKiPbQGAMhhBAJOmPoxPjFc4HLdS0XxHeB/oWNygfUAdSIh2iBHLfelvv23YFQYujMLtcBLmfg5zXG14in+fMRiIWQQJrfFqPuEo9amzuk22LIsQ0WdCmJEEIUcLLM6ftABzyD1y6XiJNlAb6chXEbLCgxEEKIAuS49Xa4bt/dXNBLSSdOnMD+/fvx008/+e7S17t3b1x33XXIysqSNRhCCIlWMWYd6i7xkg/2UG+9Lcc2WLSaGL777ju8//77sNvtGDhwIPr37w+LxQK73Y6zZ89i7dq1sFgsmDFjBoYOHSprUIQQEm365phQa3O3uPV23xyTottg0Wpi+PzzzzFv3jz069ev1RcfO3YM27Zto8SgVfFdWp2VpDVcWnrAgWZqxEPUQo5bb4fj9t2BUKOeDtBKYxFAO7FSnPLSSpyAdmKNhjhlbdTT2NgIh8MheS4lJSWUTRAVkWPuf7BtUH0BIdrDlBgOHjyI1157DRcvtuzEtWXLFtmDIsqQY+5/sG1QfQEh2sOUGDZs2ID/+I//wJgxY2AyyTvIQQghRF2YEoPL5cJNN93Urvt6E0II0RamT/rbbrsN27Zt00zHIkIIIe3HdMYwYsQIPP/889i6dSsSEhIky9atWxeWwAghhEQGU2J46aWXcPXVV2PUqFE0xhBF5Jj7H2wbVF9AiPYwJYbKykq88MILNMYQZeSYLhpsGzQllRDtYUoMw4YNQ2lpKQYPHhzueKKCEnP3Wfbhv06tOQa8o0n2OAhRAyV6FHQmzLOSXnzxRQwYMACJiYmSZY899lhYAtMyJebus+zDfx3eYADcbtnjICTSlOpR0JkwJYaePXuiZ8+e4Y6FEEJC1laPgoFDLRGOTpuYEsO0adPCHQchhLSLUj0KOhOm0eStW7fi2LFjkue8d1YlhJBIijHrWtRYhaNHQWfCdOR27tyJHj16SJ7r0aMHdu7cGZagCCGEVd8cE4xGzpccwtWjoDNhupTkdrthMEhXNRgMcDrl7TMaLZSYu8+yD/919OYYuP1mJRESLZTqUdCZMCWGzMxMFBYW4rbbbvM99+mnnyIzMzNsgWmZElNBWfbhv06SRu4hT0h7WGL1NNAsI6bEMGfOHDz33HMoLi7GVVddhQsXLqC2thZ//vOfmXbidDqxdOlSuN1u8DyPkSNHYvr06ZJ1du3ahXfffdfX3+HWW2/FxIkTQ3w7xB+/8kmg2nOr9It6PQSe9yxI7Qr9ouWS5RK/LAeUr8mgegtCIo95uurLL7+Mffv2obq6GiNGjEBubi7MZjPTToxGI5YuXQqz2Qy3242nn34aQ4cORU5OjmS90aNHY+7cuaG/CxJY9UWg/hIAQOA4oPlNEP2Wt0bpmgyqtyAk8pg7uJnNZowZM6ZdO+E4zpdEeJ4Hz/MtppcRQghRh1YTw6pVq3DnnXeiX79+rb742LFj2Lp1K5544omgOxIEAYsXL8b58+cxadIkZGdnt1hn7969OHLkCNLT0zFnzhxYrdYW6xQVFaGoqAgAsGLFioDrKMVgMER0/8Fc1Os9ZwoAOADiL//W6fWwWq2S5f68y4FfLu0YWv6a6M0xSJLpvfvvg+Pgm+gg5z7kpvafvZdW4gS0E2tniLPVxJCXl4eNGzeisbERAwcORPfu3WGxWGC323Hu3DkcOnQIcXFxmDFjBtOOdDodVq5ciYaGBqxatQqnT59Gr169fMtzc3MxZswYGI1GfPbZZygoKMDSpUsDxpWXl+d7HMkBVbU3BRd43nf5SPS7lCTwPKqqqiTLm7/O+754R5Pv0o4/t6NJtvfuvw+DwQD3L/+Wcx9yU/vP3ksrcQLaiTUa4uzevXubr201MQwdOhRDhw7F8ePHceDAAZSXl6OxsRFxcXHo3bs3Fi5ciL59+4YcbFxcHAYNGoTvvvtOkhj8+zxMnDgR7733XsjbJoQQ0nFBxxiysrKQlZXVoZ3U1dVBr9cjLi4OTqcTBw8exNSpUyXr1NTUIDk5GQBQUlLSoqCOtENqV98/dc1mJTVf3trrlK7JoHoLQiKPefC5I2pqalBQUABBECCKIkaNGoXc3Fxs2bIFWVlZGDZsGD755BOUlJRAr9cjPj4e+fn5SoQW1bxTToHAp5X+y1ujdE0G1VsQEnmcqPFGzhUVFRHbt1auNQLaiVXtcYqi5y6e3ji9j9VK7cfTn1ZijYY42z3GQNpPjqIwluKzULYRqMAt1GY/7XkvcrwPtSgsr4XdzWPq1Z4iTFEUse1HGywGPSZlJzFvh6WpDDWeIZFEiSEMZCkKYyg+C2UbgQrcQm3209o6rDFomSiKsLt5FJ+qBwDMtVqx7Ucbik/VY3yfBOYzB5amMtR4hkQac2KoqKjAqVOn4HA4JM9PmDBB9qAIURuO43xnCsWn6vGvM9/D7XZjfJ8ETL06hflyEktTGWo8QyKNKTH8/e9/x0cffYTevXsjJiZGsowSA+ksvMnBe9YAIKSkALA1laHGMyTSmBLDzp07sWzZMvTu3Tvc8RCiWt4xBX/bfrSFlBxizDrUXZLeEqZ5UxmWdQgJJ6bfNJPJhIyMjHDHQohqeZOCd0xh44whGN8nAcWn6rHtR1uLDmKtYWkqQ41nSKS1esYgCFdOW++55x68+eabmDZtGhITEyXr6XT0LaY5WYrCGIrPQtlGoAK3UJv9tLYOawxMz6sUx3GwGPSSMQXvmIPFoGc+Y2BpKkONZ0iktVrHcM899zBtYMuWLbIGFCqqY2CjlVjVHqdSdQzNt9ve/aj9ePrTSqzREGe76xjWrVvXsahIxAVrgKNEE55oE2hQWG6F5bWwN/LIFi1wOkSYzBzKOTsssaHVS3RUTbUTP+xrgqtJgDFGh2tzY5CcSpezOoNWE0PXrldO9T/++GPccccdLdbZsWMHbr/99vBERjosWAMcJZrwkNCIogh7I4+6cgHHjU1IshhQe8GNOpcAZLf/zCFUNdVO7PmqETwP6DjA7uCx56tGjPwVKDl0AkwDBB999FFIzxNC2ofjOGSLFiQZDah38vj5UhPqnTySjAZkixbFbr/xw74mX1IAPP/nec/zJPq1OV21tLQUgGcg2vtvrwsXLsBioWIbQuTmdIhIsngSg1eSxQCnQ7lzOVeT4EsKXjoOcDmplqIzaDMx/OUvfwEAOJ1O378Bz7eapKQkPPjgg+GNjpBOyGTmUHtB2hyp1u5GalpMK6+QnzFGB7uDlyQHQQSMJpqF2Bm0mRgKCgoAeAaiH3vsMUUCIqQzE0UR5ZwddS4BSSaDZ4zB7katy41yTsBAhS4nXZsbIxljEERAr/c8T6IfU+UzJQVtCtYAR4kmPCQ0HMd56hWygSwxBk6HiNS0GJRzAiyx7PUSHZWcasLIX3nGFFxOAUYTzUrqTFpNDI888gjTBvwvMRF1CdYAh6aktiRX/UBHTMpOarHf5mcKSsSZnGrC+FvCnwjUcMyJVKuJYf78+b5/Hzt2DF999RV+/etfo2vXrrh48SIKCwsxfvx4RYJUkhxz+4NtQ6keBcHqGIiUf78FjuPa3W9BDm3VS7DUOXj7OUC8AHDOsFVOB+sbEWy5kj0uCLtWE8PAgQN9/964cSP+9Kc/ISUlxffcddddh2XLlmHKlCnhjVBhcsztD7oNhXoUBKtjIFc077cw9eqUdvVbCDeWOgeHXfD1czAaAZfLFZZ+DsH6RgRbrmSPCxIapjEGm80Gs9ksec5sNsNms7XyCkK0pXm/Be+HVaj9FsLNW+dw3Oipb/BOaU0yGZAlxoDjOMX6OQTbT7DlSva4IKFhmns2bNgwvPDCCzh48CDOnDmD77//HqtWrUJubm644yNEMf4fVF5qSgpe3joHf/51Dkr1cwi2H5Y45Djm1L9CfkyJ4Xe/+x1ycnLw+uuvY/HixXj99dfRr18/PPTQQ+GOjxDFtNZvgfWW2koxmTnU2lvWOZjMng/HGLOuRczh6OcQbD8scchxzJV6v50J06Ukk8mEWbNmYdasWeGOh5CIaN5vwX+MAVDPmQNLnUPfHBNqbW64XOHt5+C/H+9gvf9+gi1vfsznjs3Bxq/LQj7mwfZDQtdqYjh8+LBvALr57TD8XXPNNfJHFUFyzO0Pug2FehQEq2MgV8jVbyHcWOoc/Ps5QDQBHMIySydY34hgy5XscUFC02o/hj/+8Y9YvXo1AODRRx8N/GKOi/jtuakfAxutxBrpOFnn1HeGOAVBkDTiav5YrjiV6nEhl0j/7FmFpR+DNykAV26NQdhEUy1EZ6NEvwU5OOxCWL8hv7S7Ak67gCnWFF+txPYqG0wWHR4f0/aHildheS3sl3ik2IxwNYkwxnCwpbhgSZTWKGjlmHcmTGMMJSUlGDBgAOLi4sIdT1SIploIoj7hnrcvCAKcdgGpVUaU1tqR3sWIc2dcSHUbUW11MZ05iKII+yUewnEOteBh0HNoaBQg1HCwZ/GqPyvo7JgSw/bt27FmzRqkp6dj4MCBGDhwIAYMGIAuXbqEOz5CSDPhnrev0+kwxZqC0lo77G4BJ2yesakuBgNGWxOYLidxHIcUmxG14CEAcPKerzQGcEiyGSgpqBxTYnj22WfhdDpRXl6Ow4cPo7CwEOvWrUNaWprkkhMhJPyUmLfvdIhI72L0JQUASO9iDKknhKtJhEHP+ZICABj0HFxOdU3/JS0xjyQJggC32w2XywWXy4W4uDhkZGSEMzZCSABKzNs3mTmcq3NJnjtX5/LVSrAwxnBw89I43bwIo4nOFtSO6YzhySefRG1tLfr374+BAwfi4YcfRo8ePcIdGyEkgHDP2xcEAdurbEh1G9HFYPCMMdS5UOd2Y3uVDQMEM9MYgy3FBaGGgwEcDHpPknDD8zyNMagbU2KIjY1FVVUVGhoafP/xPA+9nuYJBxJNtRBEfcI9b1+n08Fk0aHa6sJoa8IvtRIG36wk1jEGS6Ie9iweSTYDXE4R8Sadb1YSJQV1a7WOoTme53HixAkcOXIER44cwbFjx9CrVy/8+c9/DneMbaI6BjZaiZXilJcW6hjkiFVJ0RBnu+sYmrPb7aipqUF1dTWqqqrQ2NgIp9PJ9Fqn04mlS5fC7XaD53mMHDkS06dPl6zjcrmwbt06nDhxAgkJCVi4cCHS0tJYwyMqJkcjFpYPKTn2o5ZtKCFYnHLUF1CNgjYxJYYnnngC58+fR1ZWFgYMGIDZs2ejf//+iIlh6/9qNBqxdOlSmM1muN1uPP300xg6dChycnJ863zxxReIi4vDK6+8gt27d2Pz5s34wx/+0L531QqWwjM5itPkEKyALdT3EqlGPXI0v3lpdwUaXTyeGp8BnU4HQRCwrPgsYo16X7GVHPuRowEOyzaUcP6sA99/2wSBvwSdHhhyQwy6ZVy5dX6w48X6PpRokMOyDzU06tFKnCyYEsMDDzyA7OxsmEztG9ziOM7Xz4HnefA83+KbQ0lJCaZNmwYAGDlyJN58803Zv2mxFJ7JUZwmiyAFbKG+l0g06pGj+Y0gCGh08Th4vhHLis/iqfEZWFZ8FgfPN2Jwt1gIgmfqZkf3I0cDHJZtKPGN+fxZB77d7bjyg3YD3+524IYxQLcMc9CfiyAITO9DiQY5LPtQQ6MercTJiikxDBo0qMM7EgQBixcvxvnz5zFp0iRkZ2dLlttsNqSmpgIA9Ho9YmNjUV9fT0V0GiZH8xudTidJBjM+KAcADO4W6zuDANDh/cjRAIdlG0r4/tumltlf9DzfLcPM9HNheR9KNMhh2YcaGvVoJU5WzGMMHaXT6bBy5Uo0NDRg1apVOH36NHr16hXydoqKilBUVAQAWLFiBaxWK/Nra80xnm/OzejNMUj6ZTss63gZDIaQ9h+Ki3o9hAAfJDq9HlarNeT3wnGeeJuvo4S5Viv+deb7K4/H5rT6IdnaMX3pN1bk/WWP3+MhLcYYQtlPIJx4AdYEDpdtjb7nrAlmcKLJE5N4AUbjL+tynO94wrucZRsKEPjAZ5oCD0kMbR0vpvfhdzwk2vleA/7sWfYhcxzBaDpO1tfKHEtQcXFxGDRoEL777jtJYkhJSUF1dTVSU1PB8zwaGxuRkJDQ4vV5eXnIy8vzPQ5ldgDvaPJdTvHndjT5tsOyjlc4ZycIPA8EmDAm8DyqqqpCfi8GgwHuX/4d6L2Ei/fatdsv1o1fl7X6TT7QMfWOKfgfj8f//r3kjCHU/QSMlWtCVX2TpHisqt6BxBTRExPnhMvl8iUFt9vtWZe78nsYdBsK0OkBtPzVgE7vF2eQ48X0PvyOh1fz4xGKgH9PLPuQOY6ojvMXwWYlKdLiqK6uDg0NDQA8M5QOHjzYomo6NzcXu3btAgDs2bMHgwYNohkMGte8EcvqW3tjfJ8EFJ+qZ+7S5U0K3jGF96dnY3C3WN+YgyAIsuzH2wCn1uVGgkmPnokxSDDpf2mAY4coiuibY4LRyPm2F6jxTLBtKGHIDTFA8z8d7pfnEfznIggC0/sIdjzkwLIPJeKIljhZtXrGcOHCBaYNXHXVVUHXqampQUFBge+PeNSoUcjNzcWWLVuQlZWFYcOGYcKECVi3bh3mz5+P+Ph4LFy4kPlNsGIpPJOjOE0WQQrYQn0vkWjUI0cjFp1Oh1ijXjKm4B1ziDXqfWcMHd2PHA1wWLahhG4ZZtwwBr/MSkKLWUnBfi46nY7pfSjRIIdlH2po1KOVOFm1WuB2zz33MG1gy5YtsgYUKipwYxPJWOUocmpeaR+o8l6JgiyWpjLB4lCyVqKtn7sScYZCK39P0RBnuwvcIv2BHylqqWOIJh0tcgo05357Wa2kRiHQOh8frQmpjiFYrP77AK5ckgklDtZ6i7bmu8tWsxHkeEVTcZpW6gfUQpExBi3xzf1v9l+gZEHCz3/OvXe8wHt93O72NHxhWUcNcQiCwBSnd777xQsu1NcJuHjBhQN7GmFv5GV5r0ocLzVp63iSwJhmJfE8j8LCQhw+fBj19fWSZc8++2xYAiMEYK+F6GgdQ6hx/OvM93C73SHHwRJnsPnuctRshPt4qYmW6gfUgumM4e2330ZRUREGDhyIEydOYMSIEbh06ZIshW+EBOP/QebV/AOMZR01xMGyjWCNeOR4r0ocL7VQorFRtGFKDHv37sVTTz2FyZMnQ6/XY/LkyVi0aBEOHToU7vgI8V3q8Nd8GirLOmqIg2UbwRrxyPFelTheaqFEY6Now3Qpyel0+m5XYTKZ0NTUhIyMDJw6dSqcsRHSYs69/319gCuXboKt09Fvws3jmDs2Bxu/Lgspjjv6J+PjozVB42yrEQ/L8WC5L1S4j5eahLuxUTRiSgwZGRk4fvw4+vXrh8zMTHz44YewWCxISUkJ/mKNUU0dAwHAXgvR0ToGJeLQ6XRs2wgy312Wmo0wHy810VL9gFowNeo5duwYdDodMjMzce7cObzxxhuw2+2YPXs2BgwYoEScraI6BjZaibW1OFnm1LPUOnQUSx1DsDiUqGNgiVPJegoWWv8dVZuwN+rp16+f79/p6ekR79pGOh+HXWjzG1+gng3L/69C0rNBjrnsweb2s8QR7L2wxFprc+GHfU1wNQkwxuhwbW4MklM9l0ZY6i1Y3kswctRTqIlaekuoAfNN9EpLS/H111+jpqYGycnJGDNmDK699tpwxkYIgOD3sWfp2dDkEMN+L3y54gj2fmuqndjzVSN4HtBxgN3BY89XjRj5KyApxSjptTDXag25NwULOXptqIlaekuoBdOw/Pbt27FmzRrEx8fj+uuvR0JCAtauXYvt27eHOz5C2pyHDlzp2eC9ud6MD8p9H8beb+7BtiEHueIIts4P+5p8SQHw/J/nPc97xwu8N8Wb+/73kkFmuT6sm+/nj//8KSz7UYoSvx9K7EMuTIlhx44dePrpp3Hfffdh0qRJmDVrFp5++mns2LEj3PERwjQP3fuh7M//ttxKzWWXI45g67iaBF9S8O2XA1xO+eocWERTLYQSvx9aqqdgnsjbrVs3yWOWu6oSIgeWeei+ng1+vLflZt2GHOSII9g6xhgdhGZTRgQRMJrkq3NgEU21EEr8fmipnoIpomnTpmHDhg04d+4cnE4nKioq8Oqrr2L69OkQBMH3HyHhEOw+9iw9G5S4F75ccQRb59rcGOj18CUHQQT0es/zzWsUNs4YEnJvChbBejpoLTmopbeEWjANPr/++usAgN27d0ue//rrr/Haa6/5HnfWO7KS8Ao2D52lZ4MlFmGfyy5XHMHeb3KqCSN/5RlTcDkFGE3SWUlK1ChEWy2EWnpLqAVTHcPFixeZNta1ayvNZcKI6hjYdCRWJe/L35E6hmD9BeTs19CRfgxKYIlTzv209jgUWvl7ioY4Zalj8H7gC4KAS5cuITk5OcQQiVapYa46awzNP3z9H7+0uwLCZRHDuQS4mkQYYzj8W6yHLp7z1ReEEgfQen1AW3GwCjbfPdjyjtYosFJqP0RZTL+xDQ0NePnllzFr1iwsWLAAAFBSUoL3338/rMGRyFLDffvliEEQBAiXRfSpMcNWw6OpSYSthkefGjOEyyLT+JiSxyJY/wDqL0DCjXmMIS4uDuvXr8fjjz8OAMjJycE777yDGTNmhDVAEjlquG+/HDHodDoM5xJg4zwf4E1uz4e4gfM8z/KNvnkcrfVjkEOw/gHUX4CEG9MZww8//IAHHnhAcgmpS5cuuHTpUtgCI+qghrnqcsTgahJh0kvXN+k5uJzs3/SVOhbB5rtraT480SamxBAbG9uic1tVVRWNNXQCapirLkcMxhgOTl66vpMXYTSxf6grdSyCzXfX0nx4ok1Mv0kTJ07E6tWrUVpaClEUUVZWhoKCAtx8883hjo9EkBrmqssRgyAI+LdYD7fo+aYdY9CB4zi4Rc/zrGMMStQHAMHnu2tpPjzRJqYxhqlTp8JkMmHjxo3geR5/+ctfkJeXh8mTJ4c7PhJBapirLkcMOp0OungOp+DwzEpyiog36fBvsRG6eI55jEGpYxG0H4OG5sMTbWKqY1AzqmNgo/Y6hmDz7lliCLaOHP0alKoPkEtn+R1VUjTEKUsdQ2lpKdLS0pCWloba2lq899570Ol0mDlzJpKSkkIOmGhLuOeqs9QHBIshWK1DoOXby2pDrsdgORZauec+UZ5WfjeYxhg2btzoO91+++23wfM8OI7Dq6++GtbgSPSToz4g2DYEQVBNDQLpvLT0u8F0xmCz2WC1WsHzPL7//nusX78eBoMBDz/8cLjjI1FOjvoAlloHpeoxqMaAtEZLvxtMZwwWiwW1tbU4fPgwevToAbPZDABwu91hDY50DnLUBwTbhlpqEEjnpaXfDabEcOutt+LJJ5/E2rVrMWnSJADAjz/+iIyMjCCvJCQ4OeoDgm1DLTUIpPPS0u8G06WkO++8E8OHD4dOp/M17ElJScF//ud/hjU4Ev2a1wfMHZuDjV+XSXoJB/tW33wb/v2HAeCO/sn4+GhNq8vlPHPom2NCrc3tu2RANQbES0u/G0yJAWg5vSnYdCdCWMhRHxBsGzqdTjU1CKTz0tLvBnNiICRcJmUnSeoBvB/coXxgB9uGHPtgZYnVq24wkaiDVn43FEkMVVVVKCgoQG1tLTiOC1g1fejQIbz44otIS0sDAIwYMQJ33323EuF1ako24ekIljg7+lhNtPJzIdFJkcSg1+sxe/ZsZGZmwm63Y8mSJRg8eDB69OghWW/AgAFYsmSJEiFFhY4Wy6ihCU/zOICWBW6scWqleCiYwvJa2Bt5ZIsWOB0iTGYO5Zwdllhlfy7RcjxJ6BQZDk9OTkZmZiYAz9TXjIwM2Gy2IK8ibelosYwamvCwxMFanKal4qG2iKIIeyOPunIBx39qQn2d5/915QLsjcr9XKLleJL2UXyMobKyEidPnkS/fv1aLCsrK8OiRYuQnJyM2bNno2fPnkqHpxkdLZZRQxOeQHEEKnBjiVNLxUNt4TgO2aIFx41NqHfyqHd6PoiTTAZkiTGK/Vyi5XiS9lE0MTgcDqxevRr3338/YmNjJcv69u2L9evXw2w2Y//+/Vi5ciXWrl3bYhtFRUUoKioCAKxYsQJWq1WR2AMxGAyR2794AUZjoOdNAWNqLda5Viv+deb7K4/H5kTkWrY3Do7zxNo8jqBxhng8OiqcP3tOvABrAofLtkbfc9YEM7h2vJd2x6nw8QQi/PcUgs4Qp2KJwe12Y/Xq1Rg3bhxGjBjRYrl/orj++uuxceNG1NXVoUuXLpL18vLykJeX53scybscRvQui5wTLperxQAluMDHJFCs3ssy/hXsG78uU7xDm38cBoMBbrdbEgdTnCEej44K589e5JpQVd8kuWxUVe9AYooY8j7bHafCxxOIjruWqklH7q6qyBiDKIrYsGEDMjIycPvttwdcp7a21veHcOzYMQiCgISEBCXC06SONmtRQxOeQHE0b4AjCAJTnNHSvEYURZRzdtS63Egw6dEzMQYJJj1qXW6Uc3bFfi7RcjxJ+yhyxnD06FEUFxejV69eWLRoEQDg3nvv9WWzW265BXv27MGnn34KvV4Pk8mEhQsX0vS8NnS0WEYNTXhY4mAtTtNS8VBbOI7zxJwNZIkxcDpEpKbFoJwTYIlV7ucSLceTtA816umASJ9ShjLXvbVY1TJf3ts0xxtn8yY6aonTK9w/e7neb6R/R0OhlVijIU5VXEoi8issrw14k7jC8tqQtqOGoq+Xdldg+f9V+HovC4KA5f9XgZd2X0n6aohTSZ3t/RJ1ocSgQWqpQZCDIAhodPE4eL4Ry4rPQhAELCs+i4PnG9Ho4n3JghCiHLpXkgappQZBDjqdDk+Nz/Alg7y/7AFEEYO7xeKp8Rm+zoGEEOXQX51GKdV4Rgne5OCPkgIhkUN/eRqlVOMZJXgvH/nzXlYihCiPEoMGqaUGQQ7+YwqDu8Wi6JGRGNwtVjLmQAhRFiUGDWpt7v/4PgmK1iDIQafTIdaol4wpPDU+A4O7xSLWqKfLSYREAA0+axRr4xm1zf8P5PEx3cHzvC8J6HQ6PDmuu6SOobPRws+NRC9KDCESPv4rxMpzAIBacwx4RxMAgEtLh+6OmYrGEmyue7A+B2oRKM7tZbWqi1MpaumTQTovOk8PkVh5Djh/Fjh/FvyZ075/e5OFWmil1kErcSqFjgdRAzpjiFIsfQ7UQCtxKiWaalSIdtEZQxTTSq2DVuJUCh0PEmmUGKKYVmodtBKnUuh4kEijxBClgvU5UMuHjFbiVEo01agQ7aIxhhBxaenw/mnqzTFw+81KUhO19FsIRitxKoWOB1EDSgwh8p+SmqTy+7Kz1jpEmlbiBOSpLwi2DS0dDxKdKDFEOa3c118LcRaW18LeyCNbtIATL0DkmlDO2WGJZa8vYK1R0MLxINGLxhgIYSCKIuyNPOrKBRz/qQm1NU4c/6kJdeUC7I1s9QVUo0C0gs4YCGHAcRyyRQuOG5tQ7+Rx2dYIURSRZDIgS4xh+kZPNQpEK+iMgRBGToeIJIv0u1SSxQCng/2bPtUoEC2gxEAII5OZQ63dLXmu1u6Gycz+oU41CkQL6FISIQxEUUQ5Z0edS0CSyQBrghlV9Q7Uutwo5wQMFC1Bv/U3r1GYenWK7zFAZw5EPSgxEMKA4zhYYvVANjxjCqIJiSkiyjkBlli2+gKqUSBaQYmBEEb+9QXWX2pYWM4UWtsGQDUKRJ1ojIGQEMhRX0A1CkTtKDEQQgiRoMRACCFEghIDIYQQCUoMhBBCJCgxEEIIkaDEQAghRIISAyGEEAlFCtyqqqpQUFCA2tpacByHvLw8TJ48WbKOKIp46623cODAAcTExCA/Px+ZmZlKhBfV5GgsQwjpXBRJDHq9HrNnz0ZmZibsdjuWLFmCwYMHo0ePHr51Dhw4gPPnz2Pt2rUoLy/HG2+8gWXLlikRXtTybwoDoNWmMIQQ4k+RS0nJycm+b/8WiwUZGRmw2aR3mCwpKcH48ePBcRxycnLQ0NCAmpoaJcKLStQUhhDSXorfK6myshInT55Ev379JM/bbDZYrVbf49TUVNhsNiQnJysdYlRo3hTmX2e+h9vtpqYwhJCgFE0MDocDq1evxv3334/Y2Nh2baOoqAhFRUUAgBUrVkiSidIMBkNE989irtWKf535HhzniXfu2BxVJwUtHFOA4gwHrcTaGeJULDG43W6sXr0a48aNw4gRI1osT0lJQVVVle9xdXU1UlJSWqyXl5eHvLw832P/1yjNe4dNtfJePnK73TAYDHC73dj4dZmqzxjUfky9KE75aSXWaIize/fubb5WkTEGURSxYcMGZGRk4Pbbbw+4zrBhw1BcXAxRFFFWVobY2Fi6jNQBzZvCbJwxBOP7JEjGHAghJBBFzhiOHj2K4uJi9OrVC4sWLQIA3Hvvvb5sdsstt+C6667D/v37sWDBAphMJuTn5ysRWtSipjCEkPZSJDFcffXV+OCDD9pch+M4zJs3T4lwOg1qCkMIaQ+qfI5y1BSGEBIqSgyEEEIkKDEQQgiRoMRACCFEghIDIYQQCU6kCe2EEEL80BlDByxZsiTSITDTSqwUp7y0EiegnVg7Q5yUGAghhEhQYiCEECJBiaED/G/mp3ZaiZXilJdW4gS0E2tniJMGnwkhhEjQGQMhhBAJxTu4aZUgCFiyZAlSUlJajPbv2rUL7777rq9/xK233oqJEycqHuOjjz4Ks9kMnU4HvV6PFStWSJaLooi33noLBw4cQExMDPLz830tV9UW66FDh/Diiy8iLS0NADBixAjcfffdisfZ0NCADRs24OeffwbHcXjkkUeQk5PjW66WYxosTrUcz4qKCvz3f/+373FlZSWmT5+O2267zfecGo4pS5xqOaY7duzAF198AY7j0LNnT+Tn58NkMvmWu1wurFu3DidOnEBCQgIWLlzoi7lVImGyfft2cc2aNeLy5ctbLPvyyy/FN954IwJRSeXn54uXLl1qdfm+ffvE559/XhQEQTx69Kj45JNPKhidVLBYS0tLAx5rpb3yyitiUVGRKIqi6HK5xMuXL0uWq+WYBotTLcfTH8/z4rx588TKykrJ82o5pl6txamGY1pdXS3m5+eLTU1NoiiK4urVq8Uvv/xSss4///lP8dVXXxVFURS//vpr8aWXXgq6XbqUxKC6uhr79++PyFmAnEpKSjB+/HhwHIecnBw0NDSgpqYm0mGpVmNjI44cOYIJEyYA8LRKjIuLk6yjhmPKEqca/fDDD+jWrRu6du0qeV4Nx9Rfa3GqhSAIcDqd4HkeTqezRYOzkpIS3HjjjQCAkSNHorS0NGijLrqUxGDTpk247777YLfbW11n7969OHLkCNLT0zFnzpyI9YR9/vnnAQA333xzi1kJNptNEldqaipsNlvEOuW1FSsAlJWVYdGiRUhOTsbs2bPRs2dPReOrrKxEly5dsH79evz000/IzMzE/fffD7PZ7FtHDceUJU4g8sezud27d2PMmDEtnlfDMfXXWpxA5I9pSkoKpkyZgkceeQQmkwlDhgzBkCFDJOvYbDakpqYCAPR6PWJjY1FfX48uXbq0ul06Ywhi3759SExMbPMaZ25uLgoKCrBq1SoMHjwYBQUFCkZ4xX/913/hhRdewFNPPYXCwkIcPnw4InGwCBZr3759sX79eqxcuRK33norVq5cqXiMPM/j5MmTuOWWW/Diiy8iJiYGW7duVTyOYFjiVMPx9Od2u7Fv3z6MHDkyonEE01acajimly9fxrfffouCggK8+uqrcDgcKC4u7vB2KTEEcfToUZSUlODRRx/FmjVrUFpairVr10rWSUhIgNFoBABMnDgRJ06ciESovsHvxMRE3HDDDTh27FiL5f7Nwaurq32vUVqwWGNjY33feK+//nrwPI+6ujpFY0xNTUVqaiqys7MBeE7DT548KVlHDceUJU41HE9/Bw4cQN++fZGUlNRimRqOqVdbcarhmP7www9IS0tDly5dYDAYMGLECJSVlUnWSUlJQXV1NQDPl4jGxkYkJCS0uV1KDEHMnDkTGzZsQEFBARYuXIhrrrkGCxYskKzjf/2zpKQEPXr0UDpMOBwO36Uuh8OBgwcPolevXpJ1hg0bhuLiYoiiiLKyMsTGxkbk9Jwl1traWt910GPHjkEQhKC/zHJLSkpCamoqKioqAHj+CJv/bNVwTFniVMPx9NfW5Rk1HFOvtuJUwzG1Wq0oLy9HU1MTRFHEDz/8gIyMDMk6ubm52LVrFwBgz549GDRoUNBOjjTG0E5btmxBVlYWhg0bhk8++QQlJSXQ6/WIj49Hfn6+4vFcunQJq1atAuD5VjB27FgMHToUn376KQDglltuwXXXXYf9+/djwYIFMJlMEYmTNdY9e/bg008/hV6vh8lkwsKFCyPSlvTBBx/E2rVr4Xa7kZaWhvz8fFUe02BxquV4Ale+DDz00EO+59R4TIPFqYZjmp2djZEjR2Lx4sXQ6/Xo06cP8vLyJJ9PEyZMwLp16zB//nzEx8dj4cKFQbdLlc+EEEIk6FISIYQQCUoMhBBCJCgxEEIIkaDEQAghRIISAyGEEAlKDKRTmz59Os6fPx9w2TPPPIPPP/9c4Yg82oorkDNnzmDJkiVB74ETqnfeecc3RZN0HpQYCIkwORLQ+++/jylTpsg+j37KlCn4xz/+AbfbLet2ibpRYiBE42pqanDo0CHccMMNsm87OTkZ3bt3R0lJiezbJupFlc9ENbZu3YpPPvkEdrsdycnJmDdvHq699loIgoCPP/4Yn3/+ORoaGnDNNdfgoYceQnx8PCorK/HYY4/hoYcewocffghRFHH77bfjjjvuAOC5VcFbb72Fs2fPwmQyYcSIEZgzZw4MhtB/9b/44gts374dtbW16NevHx566CHfrZinT5+OefPmYceOHairq8PYsWMxd+5ccBwHQRDw3nvv4auvvoLZbMaUKVPw5ptv4m9/+xs++OADHDlyBOXl5di0aRNuvPFGzJ07FwBw8OBBLFu2rMX2mjt48CAyMzMlzVmqqqqwadMmHDlyBKIoYsyYMZg7dy527dqFzz//HFlZWdi1axfi4+Mxf/58nDt3Dlu2bIHL5cJ9993nu00zAAwaNAj79+9X/Q3viHwoMRBVqKioQGFhIZYvX46UlBRUVlZCEAQAwD//+U98++23eOaZZ9ClSxe89dZbeOONNySl/aWlpXj55ZdRWVmJZ599Fn369MHgwYOh0+kwZ84cZGVlobq6GsuXL0dhYaGkExeLb7/9Fv/4xz+wePFipKenY+vWrXj55Zfx3HPP+dbZv38/li9fDrvdjsWLF2PYsGEYOnQoioqKcODAAd+dT/07g9177704evQoxo0b16LfR2vba+706dNIT0/3PRYEAS+88AIGDRqEgoIC6HQ6yY0dy8vLMWHCBLz55pv44IMPsGbNGuTm5mLt2rU4fPgwVq9ejZEjR/puEJeRkYG9e/eGdLyIttGlJKIKOp0OLpcLZ86c8d3vp1u3bgCAzz77DDNmzEBqaiqMRiOmTZuGvXv3gud53+unTZsGs9mMXr164aabbsLu3bsBAJmZmcjJyYFer0daWhry8vLadTvyzz77DHfddRd69OgBvV6Pu+66C6dOncLFixd969x5552Ii4uD1WrFoEGDcOrUKQDAN998g8mTJyM1NRXx8fGYOnUq0z5b215zDQ0NsFgsvsfHjh2DzWbD7NmzYTabYTKZcPXVV/uWp6Wl4aabboJOp8Po0aNRXV2Nu+++G0ajEUOGDIHBYJAMfFssFjQ0NIRwtIjW0RkDUYVu3brh/vvvx4cffogzZ85gyJAh+O1vf4uUlBRcvHgRq1atklxG0el0uHTpku+xtxEJ4Lnj5OnTpwF4zkTeeecdHD9+3Nflqj39gy9evIi33noL77zzju85URRhs9l8l5P8b80cExMDh8MBwDMG0Dw+Fq1tr7n4+HhJE6mqqip07doVer0+4PqJiYm+f3svP/nvy2QySfZlt9s10RGOyIcSA1GNsWPHYuzYsWhsbMRrr72GzZs3Y/78+UhNTcUjjzwi+dbrVVlZCcBzz37v7Yarqqp8t2l+44030KdPH/z+97+HxWLB//7v/2LPnj0hx2a1WvGb3/wG48aNC/m1ycnJsNlsvsf+vQYAdHgmUa9evfDVV1/5HlutVlRVVYHn+VaTQyjOnj2L3r17d3g7RDvoUhJRhYqKCpSWlsLlcsFkMsFkMvk+MG+++Wa8//77vss2dXV1+PbbbyWv/+ijj9DU1ISff/4Zu3btwujRowF4vu16G6qcPXu23XPyb775ZmzduhU///wzAE+f5W+++YbptaNGjcLOnTths9nQ0NCAbdu2SZYnJibiwoUL7YoLAAYPHoyTJ0/C6XQCAPr164fk5GRs3rwZDocDTqcTP/74Y7u3f/jw4YBjGyR60RkDUQWXy4XNmzfj7Nmz0Ov16N+/v+8++JMnTwYAPPfcc6ipqUFiYiJGjRolmZ45cOBALFiwAIIgYMqUKb6+t7Nnz8Zrr72Gbdu2oW/fvhg9ejRKS0tDjm/48OFwOBxYs2YNqqqqEBsbi2uvvRajRo0K+tqJEyeioqICTzzxBCwWC37961/j8OHD0Ol0vvdXUFCAzz77DOPGjcODDz4YUmxJSUm45pprUFJSgtGjR0On02Hx4sV48803kZ+fD47jMGbMmIBnXMHU1NTgzJkzGD58eMivJdpF/RiIpnmnq/7tb3+T5bKJEg4cOIDXX38d69evl22bZ86cQUFBAZYtWyZrkds777yDq666CpMmTZJtm0T96IyBkDBzOp0oLS3FkCFDcOnSJfzP//yP7N/Ae/TogeXLl8u6TQD47W9/K/s2ifpRYiAkzERRxIcffog1a9bAZDLh+uuvx/Tp0yMdFiGtoktJhBBCJGhWEiGEEAlKDIQQQiQoMRBCCJGgxEAIIUSCEgMhhBAJSgyEEEIk/h+etkSio0GtdgAAAABJRU5ErkJggg==\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 可视化特征\n",
    "marker = ['s','x','o']\n",
    "for _,c in enumerate(np.unique(y)):\n",
    "    plt.scatter(x=iris_data.loc[y==c,\"sepal length (cm)\"],y=iris_data.loc[y==c,\"sepal width (cm)\"],alpha=0.8,label=c,marker=marker[c])\n",
    "plt.xlabel(\"sepal length (cm)\")\n",
    "plt.ylabel(\"sepal width (cm)\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以看到：每种不同的颜色和点的样式为一种类型的鸢尾花，数据集有三种不同类型的鸢尾花。因此因变量是一个类别变量，因此通过特征预测鸢尾花类别的问题是一个分类问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "各个特征的相关解释：\n",
    "   - sepal length (cm)：花萼长度(厘米)\n",
    "   - sepal width (cm)：花萼宽度(厘米)\n",
    "   - petal length (cm)：花瓣长度(厘米)\n",
    "   - petal width (cm)：花瓣宽度(厘米)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 无监督学习"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以使用sklearn生成符合自身需求的数据集，下面我们用其中几个函数例子来生成无因变量的数据集：   \n",
    "https://scikit-learn.org/stable/modules/classes.html?highlight=datasets#module-sklearn.datasets    \n",
    "![jupyter](./1.2.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:34:32.719213Z",
     "start_time": "2021-03-14T11:34:32.578215Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAACDOUlEQVR4nO29d3xc1Z33/z73TpOsOpK7JKuY5hBwEqdBQgn+5cmTzS5+kjxJNo2EAAu2IdimuALBgE2xIYCd0EKyZDeb15MnDVKejTeLCbBkIdgU2xiruchykaaoTb/n98eZe6do5IJtFeu8Xy+wZubOzNHVved7zrd8vkJKKdFoNBrNuMUY6QFoNBqNZmTRhkCj0WjGOdoQaDQazThHGwKNRqMZ52hDoNFoNOMcbQg0Go1mnOMa6QG8V/bv33/Kv6O6upqurq5T/j2nAj32kUGPfWQYq2Mf7nFPmzat4PN6R6DRaDTjHG0INBqNZpyjDYFGo9GMc7Qh0Gg0mnGONgQajUYzztGGQKPRaMY52hBoxhVSSmQsihbd1WgyjNk6Ao3meJFSYm24G3a3wIwmjAUrEEKM9LA0mhFHGwLN+CEeU0Yg1J157PWN7Jg0mlGANgSaEUNKqSZjj9dZmUspkdEIUsqTv1r3eGFGk/p5RpN6rNFotCHQjAyF3DQA1oa76drXhlXTcNJdN0II9T15xkejGe9oQ6AZGfLcNDIWVc+1N2OFA5CyTonrRgih3UEaTR7aEGiGjWxXUI6bpq4J+cQDsKcVpIXwVyNrG7XrRqMZJrQh0JxynJRNe7JPu4JsN42UErlqvtodVFRRve7HdCdSJ+y6KRSD0Gg0g9GGQHNMHOukmn9cJhbQDH29kEyoA+MxhNcHXh/SssBKgRDq37IKRCBwwuMtFIPQhkGjGYw2BJqjcqz59zmTft1MjIUrsmIBAXC5odw/KGNHJOJIYYCUIAzoCb3nrCHbEEkpB8Ug5JPrdA2BRlMAbQg0R+cY8u+llFg9IXj7dUgloW+LCgB7vFBbrw6qa0JcvQTh9eVMwtLtgdqG9AOLriXfQtY25kzWx7IjyTFYdY1Ql5UqCrqGQKMZAm0INEfnKPn3zgTc3gypVM7zbLwH9rRBbQNiwXIMwxj0XrnxHtjXDlNroWM3MtAFlnQm69wJPteY5ASg8wyWWL1RHeP2qNfqGof8HTSa8Yw2BJqjctT8e3sCDqfdP8UlUD9TTcL280IoF5A9Ydufkz15hwPqvZVVajVvT9bZx/T1IFdeh6yfiZi/XBmRtLtHzF8+yGDJeCxzTF2TMg55O5KhcNxMbg8iEdexBc1pizYEmiEZ5I5Ju1IGPZ+XCmqv2AFk1sQs3R5k1sqeqxar12rq1SQvJfT34pr5YZLfWZwZh9uTCSYnk8pg7G6Bvp5B7h5x1RL1s8erDEB7M/RngtRCiFy31BAup5xdjrSQhqljC5rTFm0INAUZKkA81PPZOwZIF4gBYv5yZzVNPKZ2COmVPTd+DQwDznwfmC4VWwCS27bCqvlY9TPV52cHk00XlJSp2gO3J+PuqW1APn4f7G1Xu4OrlmR2KQhlRFIptbo/wu8I5Aab07sZbLVSHVvQnIZoQ6ApTKHKX9snbz8vJbI3DKXlzo5BSon16F2wfav6nLPPQ159EyIWRbrcMK0GekKZNFIrpSbvs86FHW+oCTeZUBNwe7P6/JIyqJ+Zu5N4ch3ctkA9vnMDPH4/bHs9M2GD2qVIqT5LAgN9apL3+o6eWVTXpIyMlCAtSO8IdGxBczpyUgzBxo0bef311ykvL2fdunWDXpdS8vTTT7Nlyxa8Xi/z58+nsVGt5J5//nl++ctfAvD5z3+eSy655GQMSXOcDMr/d3uUy0ZKqJmBfHKdWs3nTZBy9SJVBXzFQkRpuVr9727JTPRv/w2++4/KtSLICSY79IbUexCABJcLikvBUp9v+//tnYUVjSiXTU8QSKef7m1Lp58KqK1XcYAFK7DCQbj5W+p7rBQylUI+shr2th01syg72KxjBJrTmZNiCC655BI+85nPsGHDhoKvb9myhQMHDvDwww+za9cunnzySe655x76+vr4xS9+wdq1awFYunQpc+bMoaSk5GQMS3OM5LtInCDs3jblrmlvgehAZnK/c4MqDrv3VjVphrrhpiuQZRXI+55WE+xbQdQyPI1VwABk098LCCitAENkVvCpJEiJSMQRXh+WZcHKa9WuAmBqrTJaM5pgNyo7aeEqZ8IWviK1E0km1Kr+B/fAzredr82e7GUsmkl1ndHkBJUFaHeQ5rTmpHQomzVr1hEn79dee42LLroIIQRnnnkm/f39BINBtm7dynnnnUdJSQklJSWcd955bN269WQMaVxwIt22ct6b7Qayg7Dtzcql0tcDfWH1pnK/miifXKeMQF9P7of2hKCvB3HNzVBeeeQBGObg50wTlt6LME212k8l0/55S032oL7TNgIAO7bCqvmAgNUb4ZqbkbEoVlrKGo8Xzn6/MjBeX44RYPoMFdT2eFUQe/E3YPsbUFOvdiBHWf3rbmea04VhiREEAgGqq6udx1VVVQQCAQKBAFVVVc7zfr+fwBDSAps2bWLTpk0ArF27NufzThUul2tYvue9IKUktGYpydZ3cDWeRcWyezN59bEopmkWHLut9x9+8A6SrTtxNZ5F+dK1hM84Rz2un0nZjAbCTWeReP2/nPcZs2ZjulwkWneqKuEhMP/1h/hX3E/4zFkkW3di1jSQsiyst/+W67+3UirVNL3iB0CA8YunkamUeiAAKTHcbqrLShG+Iiy/n8Ml5RnjBBAOIPa14v7J94m/+TdIJJCmiTn7IxiGQbJzL9Q2YG3fkjNWt8tFZXU1Mhrh8PatmcyijnaqPS5EeWVuQVssCln1C6E1t5JseQdX09mUL12LiMeGPO9jgdF8vR+NsTr20TLuMRMsnjt3LnPnznUed3V1nfLvrK6uHpbveS/IWBRr13YIdRNPWXTt71D+87SLx3PGOSSuukmt9sFJ53RSItMplfGURXfnfuRVNyEfWU28eSddq5fAlYvgzdecydHatR3L401n4WRhZ/Ok3UCprf9N1/4O5HeWwCN3YrU3K7fNrA/A9i3qWMOAknKoawCEckGlx2O9+VrajSRBqmwfK57gULgHo7s74wba9npmDKYLOW0G8eadkIir51JJku9uV98VDkD34UHnMLG7RY1VyhwjJZNJDi+6IlObEI9lBPPqGhFX36Tes+Wv6hz2/JXDt38X9rXjOeMcklffPCZjCaP5ej8aY3Xswz3uadOmFXx+WAyB3+/P+WW7u7vx+/34/X62b9/uPB8IBJg1a9ZwDGnsU6jaN8vFk2h5B/noatjxJgBy1my4aonSAbILvyoyuj8iHlMB13S2jjAM5BnvU64XgEi/mlTzOXMWdO7LuGvSAVkeS2cOSQl7BGL1RiUu99h9sH+P2gnsaVPZQKs3KneT7Zayg75SZmIFj6zG2tcO02ph1/bcMZSUwTcXwo+/n2uoaurAdA82XqDSUGsbM+6jWbMz441k4iEqsNyqYiKppCpoW3WdksTI3uHsbYVwkGTrTp1iqhlzDIshmDNnDn/84x+58MIL2bVrF8XFxVRWVjJ79mx+9rOf0dfXB8Abb7zBV7/61eEY0pinULWvzDIOrvozSLTszAR4d7fA4/epCc3lhnPOR1xzsxMQTbncmYCulVKTdv7Eb5hw5rnwbpaffefbMCEvPtTfq1b5WVk8eLyIvh7k/j0qFmFP9HtalezEwpUqG+jx+6GjHabWwI63VOqmEGo13hsaPKmX+9UxS69Sn1dcon4/KwX79qjA9YTSdDA6i1RKxR5WXacyh765EO5arD7fMJSRrK2H3a1OdhJm+nYJpWsTzjlPpb7WNaqd0Z4WXI1nkdQpppoxxkkxBA899BDbt2+nt7eXa6+9li996Uskk6o46NOf/jQf+MAHeP3117nhhhvweDzMnz8fgJKSEr7whS+wbNkyAL74xS/qjKHjIL/bVrZxqJg6jcPfW6zcMaAmtT1tGcMgrVy9ng13Z4K/kQGVo29PnoapJtOGM1Rlbz79fWkXkaUe/+tjyu2zB7VyvuomFYxtb4b+ntwCrVQKy7IQUsJT62FfO+7GM0gkUyp4jAlnn6c+e1uujx/TBTevgXtvyRixgT5lDAbU4oJ33hwiY0mq15IJCAfhzhtVTKKiynH/4PGqHYFdnzChFGY0ZorW5i/PdFabUIoY6KO8vnHIOJdGM1oRcoymPOzfv/+Uf8dY9TuCGvvhw4edCl/p9sDD38sUelVUYdz1A4TXp+INK67NrLbtidrlVitnIeCc8+Gam2HJNzPGZEgElJarSdMwc3L+B2G6oLQMahqU8FyoW3UoS1mZqt73fRCuvmnwd5eUQ+OZykCkChio46Xcj7jtIURpuXocj2GZLnj0Tti7G+rPQCzI1DNIKZE3fwt6w8poTSjDc+Ysklff7Lw/Ry9plNchjPXrfSyOfVzFCDSnnkKaOUIIlUcvpVqRv7tNHSwMqKnHcrkxYlEsO3sn82HpJjGWWolL1Ko4bVQANYGfc156VZ0/CUvlxtnTplbZ+UagtEI9j1Ar91C68re2XmUJ1Z9BKhZJVwRL5WYSAoomqM+1DZVpKpdXKqmM1lENVBbCyDTCASirVPGKkrJMN7XdLeocRPrTv5alzqmto9QbVkZASkcDKdHyDmS/v65RuZp0HwTNKEYbgtOAITVzbNJN4bPdQuxphVu+jSUMNQFHI/kfCjLPpSKECqqmv4f5y+HRu5Q0BOQaE5dbTfJFE9TE3xvGKTCrmQHfvUNVBD+xDvakJ0yhKout1p04uwphZJRM7ZhFtqGqPwP2CqhtTMcF2mB6Pex8K3c8dt2CZakA9413wg/WZALEdQ1w3bK0UmlWN7VsN9ae1txAcEmZ+q83k8oqUyn1HXY66o5eFUMJBzN/Cx1I1owytCEYwxTsxpXW/5Hp3GQppXq9tj430NoTzMrMyU2fLEhpOaKsArFwZY7+v7WvXU245X41we/braQp9rWr7zNNWLleZfS8nU753L8XI5lQu5WFeX2Lw8G0wJwFFVWIVQ8iSsszv0NfT6b2QAjENTdlFFDBkY12YhJWShmB+plw1ZKcFb1lWRljsbcd0d+LtAvpTJcyYHZ1M0LFO7ICwYZhkFr9A7jpm86uSJgm0k5htaltBNGutYo0oxZtCMYoOTLJ0+tUlo2dCbN6EcGZZ5P65vUqLXNPK0yZnvsBZRWZ1My+HhjKW2GYKr2zalJGbyctLielzGn24uTcS5n+XqF2KGUVyOtvy2j81M9UMYt0BzPh9SkDlhaJEwIk6r22v15JXrSndw2of+saB/cW8PpUHVraYA2lE2RFI8qtZTN9BtJ0qWC2zcr1sPwa+4yrMWY3wgH1e2a5xmQyqYLKxRNUSq2vGBaswEglR32MQDN+0YZgrGK7e8KBzEp/QqlyyYS6SbzxKqy6TmX0JBPKXVFSpib9kjJY/QOV059XbTsIkXbR/GANVr4WUV6zFwArS71TrN7o7ByEx4u44bbMin3jPUrEroCUddXUaXR37s+kxcaiuY1vyv1Ql6spNHjYwjEKhVwxUsrBAni3LchM6lYq4xqy2dOKfGQ1cl+7MoDfWKCMbDammZuq2t8DG+9GLlylajW0MdCMQrQhGCMUbAZT26BcPLZbp78X3jdbTfaJOITjauIsrVAT1zcXKndKWQUiEcfq2D34i1xu9Xm2zs8552c6jdkNYPIawtjNXpwJ2z4OBk/4Xp8KphboH2xP3oZhOIYFGLLxzbFOqEN1GpOmmXENvft2OoaSlrYoKYefP5nrMuvrUb5/K6V+vvU7mWCz6YKiCbibzibxk0eUAVZfDnvaMgZEB4w1oxBtCMYAQzWDYeFKlRJq6/iXlsM1tyoRtuwYQG9IyUHf8m1lFB74cabrF6isluJSqG+Cq2+Gpx5UAdzpdbBwpaoyzq5iTjeGAem0lCzkKgIKN4w/Sg/kfI7aKvNYzl1epzExf7lKic3uYVDuV8b1H/8JSkrh9oXKUNhNc+wMqrIKGOgfXJ9gCOUaam/OvE8IZaBtAyIlVk9IGbLjMGYazalEG4IxgIxFM24gUKtbj1dlvXTsUZk8V9wAvqLMTmFnr3Jz5OfX94aQvWG14o4MqOcME257CAwD4SuCBcsz/vwfrFE+7qyJWEqZntQAZKYBfQFXkSww4b+XiT2/eO6Yye6nnJUBJBJxuH6VkuHY255pePPUerh/maprsLOs8rOhVj4I//yoMsCpdOFbKgU9IZJv/HdWsVz2uU93SUvvJKQQyFkfwFiodweakUcbglGOlFJ1zepPS0PUNebqCtnG4ZkNyl8tLZVyeSTyV+BSwuoblY5OaTnc9cNM1s/uFuXj9/qcDmQ51bZ7xJCuImDICf89T+xHoGD/YXv3UaDTmCEE8vrbcrOg7N+jJzR4xe9yK6P7043KSJ5zntpBxaIqqJxMHCH7Kq2blG0cdjdnjPoYKDjTnL5oQzCKkXYrSLsGoMKPuFqlS1rZHcRqG9TElL3qzZ5QXK603z+l0kBtaYlZH4A96ayj7Wn3Um9YTUpDuW7iscE6QiVlQx5/Kib8oc5VIfdZ9u6jUAZR9vhkttEYJFQnYM0TKuV11XznXAvDQP50ozrE5UYUT0Bm90s4EnUNqpjv4TudbCodP9CMBNoQjFJyfNuWlc7Tb0T2hJVA3A/Xqh3AtBr4zmL40YNqtdnfl1sIJoRyEdmTi2Eq11BZBUY6h99yueHmbyuN/5Jy1XJyKNeNx6ty8tubnW5ghmEg36MP/6SR12M5u3DraBlEkNlNiPnLlfG95cq8HYFUn+H1Ie1WnXY8xG7NWVZJ5YZ/I3Ddl9S5tGMLOYi04WyA+SuUxpO9u2pv1gVnmhFBG4JhpqD7ogBWNKIKsGz5hNu+D9+7AVZcoyYY20/fE4RFX1MuD2llXBMulxJfk1K9lkqpmoFwEG79Dtb7Pqj8014fhpRYDWcoV0XdzIxrp8CENJR/f7hW/kNynAHobAq26jz3g9C2SwXaQa32s2sY0hIZ0u3JKKRKqZRU7/uRcq1tvCcngyr9ber4vbuVEdjXnrW7ajiucWs0JwttCIaRIbN/ChzHE/dnVpPJBNxxfWaFale6ZpO9ei2rxH3mLJJX3KAm60Q83ev3ukwLyD1Zq894TO0uwkGlq3+UVemIT/oFOJHMovzdhEjEEQtXKs2hx+9X5ybdwzhzbABoUdXIhpnuY9BP14Ivq5hJMitrKD9uMNCX0VCqa1DP2Rla2i2kGQG0IRhO8iYcGYuqiSJ/4orHVCYLpF06ooCU8hBBSXu3AJn0RHvVXz8zoz5aNzOz+jyB1fRo4j0bqAK/vyPYd/2qHONi2Wm3acE6OaE0/V6p9Imy23jmu4UmlKXjNZajocS1S+GRO9Xf+wdrkDpGoBkBtCEYTnIKoxqRT64bVF076LhUMkfUjKIJEB0YvMosLlWrypZ3oDdE4o1XEbGoSgclvWJOr3KBnBz241lNH6trayxxpN9/UM+HRDythSRBGBjJBCxYkVEs3dsKPeHCsthuN2LVg8gJpWonMaFUuYfsOhAdI9CMENoQDCM5GSy2yFre7sDJbLlumeqzu+q63A9xe9Sqsq9H+f6LJihXQ6QPWt/J7BzShiJ/4rYNQ6GxHW0COlbX1ljkmHcTdrDcVmDN3j0sXEl1aQmHDh5UfzfbgLtcajcwvQ5ZXKJW/rZG1P69OXUHlmUhLKugPhKcnoZYM/JoQzDMOBOOLbIGUNeEfOIB5J5W5W4QRtr9YKiMIfVOKK/MuBMCXfBvj2cUNiFHO8f1/g+Rympmf1Im7iNk5owXCrYIzRai8/pUBtdAf+ZNvglqh7Bti6ruFoaK1YQDKoPIjiNEBmDVdUhLKsntGTNzCs5OZ0OsGVm0IRghhtwdFAougpow7tygXv/h2lzN/HxKy6lYcT+BA50FNX3eM6dJLOFEyak9yJ6ca+uxlqbTerNdQ/29ODGd3rCSr7AZ6E83yUH9be2+BQDbt2BFI5hFxeqxNsSaU8RJMQRbt27l6aefxrIsLrvsMubNm5fz+o9//GO2bVPdseLxOOFwmB//+McAfPnLX6aurg5QbdtuvfXWkzGkMUHO7qC2UU0SQ7VcdKSdW9XEkkyotFLDVDsC0wSEo3FjpF0LJ3PiPqHMnNOV7Mk5HCC87jbVf6CvJ6sRkFSTf1+PchGZ6dvOPn9WSu326hrT2VshQKr3P34f8obbM0KD2hBrTgEnbAgsy+Kpp55i5cqVVFVVsWzZMubMmUNNTY1zzLe+9S3n5z/84Q+0tbU5jz0eD/fff/+JDmNMke/nVdo9ySGarKeZXpueJNJZKS630hayG84Xpxur79sNM5rSsgXdiPnLh/Q3vxdGY+roiOLxZpr+SEmyfRcsewCWXpU5xuWC+jPVxL5rW8bY213TXG6YMVN1fOvrgR89lJEH37dbFbiVlmtDrDllnLAhaG5uZsqUKUyePBmACy64gFdffTXHEGTz0ksv8aUvfelEv3ZMkd/MJNvPy3XLchVEC2G6AJGZ9EE1PIlk+aHrGhFp3RwpJaE1S7Gad6j8d+1LPmUIIRALVzkife6ms4l7fbkSH8mEmtizC/4gYxBcblUd/oM1aRdTA5z7QZVSmkoi77wRmS0/keWW0kZBczI4YUMQCASoqqpyHldVVbFr166Cxx4+fJhDhw5x7rnnOs8lEgmWLl2KaZpcfvnlfOQjHyn43k2bNrFp0yYA1q5dS3W6FeOpxOVynfD32JNysvUdXI1nUb7oDrr3tWGFukGAe+PdJOzc/swX4//h/0V6vPTct4LUti1KL9+mvBJzxkxS9vvcbqqX34tRVKy+q2UHiV7Vk8AwDapLS4bMFhqNnIzzPlxIKVV/he99H2JRTJeLlOkiNPsjJJp3IKQEw0AG8yuMsxCC6tISAunrQhgCd8MZJA2BNTAAiQRibytVbhNRXunsIrOvq4pl956wMRhL5z2fsTr20TLuYQ0Wv/TSS3zsYx9TZfhpNm7ciN/v5+DBg9x5553U1dUxZcqUQe+dO3cuc+fOdR53dXWd8vFWV1ef8PfIWBRr13YIdRNPWRzu6oKpdZBIQjJJ4s3XCrxLEIxEEdEY1r42tZK0MV0wvZ7U7haVOmoIqJtJdyAABJDp78LtgXI/Vk0DXb19iL7+At8zOjkZ5304yAkUp/szGB3tWNMblJvPkkqX6IrrYdnVgwP7dm9mr4+u3j6YXg8pCzltBvGWd9Xf0eWGskpkMsnhRVc42ULEYznXVdf+jtxmPu+BsXLeCzFWxz7c4542bVrB50/YEPj9frq7M6ud7u5u/H5/wWNffvllvvOd7wx6P8DkyZOZNWsW7e3tBQ3BWEW6PcqHDGqyeHKd0pepnQE7t+Ue7HKryX1GWmpaqEme3tdVaqjLBWe/H955KxMsXvME/HSjyjqqa1QN5AHPGeeQ+Ob1uvnJqSQni0fJTFvhgJKXsFuEvvOmcuXM+oDK9LL7ONTUQsc+pWUUjag2mbX1sGKdajC08Z60smwj/OM1cN/S3GyhAoFj7SrSvFdO2BA0NTXR2dnJoUOH8Pv9vPzyy9xwww2Djuvo6KC/v58zzzzTea6vrw+v14vb7aanp4edO3dy+eWXn+iQRg2WZSnf8Z5W5fe9arG64UPdTrcqB9OlJo7eEGzbinz4TrjhNoyF6apVKTOB5cXfyLxPiMxk1NejhObqGilfupZAIF9KWXNSyWuhqXYEJtbUOmUA0qg4QpbS64a7lc7QQLqdZSqZ6T297Go4Z7ba6UkL2ncpIyDTCrRZRWzZgWNA1xho3jMnbAhM0+TKK6/k7rvvxrIsLr30Umpra/n5z39OU1MTc+bMAZRb6IILLsi5ODs6Onj88ccxDAPLspg3b96QQeaxhpRSTebbXldP7Ezf9HXp1MKBfiieoNwD0+vg3azdgbRg2+vIR1cjrr8NI8u/L6XEmjU7o5RZWp4uTEtr3fQEYV87Ih4btt91vFJoMq4uLeFwTy9ywz1K2K9uZq7mUyyKtbctV5U0e0GQTKj3gWqOY1NWCasexCirUG+JRZVBsAPH2f2ipURmyYtoNEdDSDlkS6VRzf79+0/5d5yI/07GoljL/0lNzKBW7t//mVrVr7pOiZOV+5Ur4Mffz4jBZVPux7j7h5mOZAUqWe3HjtZNWilz0u0P5rjsxhJj1d8LmbEP5aaRUmI9cie89bfCHyCEciNBZhEBasc46wOwYDk8epfTyIbrlmV0izbek7mO0m0wgWN2F50O532scdrECDSFkW6P8tdvTxsC06VcBF4fsm4m9G1RO4M7b4T+nsIfUteIdHuQhbb82UJoWVo32kc8OjhSPweuvhkWfX2I4kGhjHl+TUkqCTu2wvfvUKnGoPol3HQFsq9HxRXu+iHcvkAtMvY0Y0UG4IkHdPczzVE5SnNbzXvBaea+b7eShiirhFmzMy6CqxaroHAqqYxAoU2ZEHDF9aoYzN7yp/sHD4VtaPTNProxfEXwvg+oHeG5c+DcD6nAv8sNhqHiRP29g9+YTGSMAMCkKZnaErvFaN1M9Tm9PbDyWrWrCAcyyqYaTQH0juBUYGeT9ATVzb5yvVIWlVK5DB6/X7UyhEx3KltJNNqf6S1cWq5e07ICpxW2JLi9e3N6U7s98NR6NWlLCxCqaLCQnhRA67vZH6paaV69BLniWnXt9fXo7meaY0IbglNBdjZJTQPc+V3o60FOKFMdqfKriKWEkjLEbQ85WvWUlDn1FlpW4PTDdh05u0e7FuE7i+HJB1RVcc0MQGR6GudjpVTsYEIJ1J+B4StSCw1b4M40oaQcausRC1fpa0czJNoQnALsbBIZiyJ/cE9Gl74vnBsUdrmhqFjdzOkMIEMISGeGZH+e1vc5TcmuRejrgVXzVVqpvQu4c4PaQbY3A5bKDLMxXcrleM3NmYQCu0c1wIQyxKoHEWmdIo1mKLQhOEkMagAjhHIH7dtT+A2zZsM/pZVWE3HnZtVFQeMMZ/eYlf7rckOFqhkw0u0y5cPfG5xZdsYsxMKVAMhH70LubVNFabXpPsjpxYW+jjRHQxuCk8CQDUM8XqUq+dZr5PQYNl1guFQDk7dViqA881xS196qfMTpFFCd5XH6k7N7tNN/6xoRV9+UCfzHY8h9uwe7h/btVq89dh+8nU5HDXXDuR9CrN6oEwc0x4w2BCeD7O29lFiRAUQyoXK7sVTTkex7OJVUvW2tVCaFcMdWVTEsRMYtoBuPjAuOlv4r3Z5cvanMK2oHubsl9+ndLZldqUZzDGhDcDLweDMVw/29sOQbyGRS5Xbbjc7zSSVVa8L850DtGOoadZbHOGNIielEHEmBSd3lVv9Or4V3Qpnna+sLXjva7agZCl1HcBw4Fbx5E7sQAnH1TaoLVTKhRMdAGYb8wiAh4Jz3q4BeupsYrnS/AZuSUuUa0DfruMR2NVor/gnr4e9hpVJqYWBfDy63SkuurVcihvv3qetICLX4sDuaFfrMlddhbbh70DWsGd/oHcExctTG4R6vCtJJlAFIJdONybNsrRAqSLxwVUZ4bEYTXLVE7RqeuF+lDdbPPGFJYc3YRcaiKnaUSqoexku+CWe9H845P6NJlUpCe0umzqCsEpbeh6iamCPz7qD7HWuOgDYEx8oQN1KOzs/2dNXn2efBV68Ff7XqOvXWa2qiLy6BBSuVEUgHBfnGAieoJ9NZRDrIp8nBriieUKKuo1QS+pPqZ5db9TuubcwxAoPcQLrfseYIaENwrAyh/25tuFvleNu9hAF2vqWkg2fMhCsXwY1fVc/396q2lLZE8dt/g5u/hSytQNbPdHYIYsGK4f3dNKMK4fUhZ81Wk38qqXaZVkrVo5gu5RayrHS/ikblVdzbjtx4D9b85SqT6IkHYE8L1M3EWLhC9zsehUgpiSRSjsT8SKINwTFS6EayohHVbCTbCEBGX377llwpYVBGIp++cEaSAvS2PY2UklhK4jXHVwaMEALj+lVqpxmNwOpF6toQAs45D665RYnJ7WlRu4J9uzPy04+m+1+EQ6jahC2OJLUuTBw9SCm5Z3MHbeE2GsrdLL+4ZkSvcW0IjoP8rA75xAOqCMjlVs3khVC9BZp3pIPGCbjj+twPKRSkKylXuww7ZqC37c6N0hKM0lTpHfEbZbhxUkq9PqyGM9JN7ZVUBPEYcscb6vrq7wNfkbr2UinY8ebQ2kSaUUMsJWkJRukeSGJZFrGUxOfShmDsEY+plZcdqLtzAyKVVLUDj652CsWcrCF7Nbdvj9rWG4bSk//mwkz1p962O9vlaNJybhRgxG+U4Sbbxz+oE1k0knuwYWYWGHbGGkBpBTScoVxNOnV0xMne4XpNQVOlF8MwaCh34zW1a2jM4aiI1tSnu431wW3zkSI9uS9cBY+sVpIAdiFQSTnccAdGMqEKhNKSwDmB4XG+bc/eLteXuWisVBNfU6V3xG+U4SQ/Q03MX65qEm2BurftHtZulYUG6UVJEqdy0TDhtofUriJb2E5XrI8I2TvcxkovSy6cxrKLplNaWUVvsDttIHD+LsPtFtWG4DjJuUlTyXQwT6pAXrrK00gmsP7plnTzEUvdlCvXq94CXh8CsJ5cB7tbkPrGdMjfLm/4XANCiHEXI8ivVHd0hGrqlfvQLjwsLnGkKGRvGPm972Y+4+zz4CcPI/e0KRXTjj2ZWJaOQQ07sZSkJRClO5KkJ5pi/rOtzPT7WPf5apa/sD/HBQoMu1tUG4LjJR4bnCUEarIvLoGaerXij0XVqg2Ue2jZ1UjDQM6arYrPdE73IPK3yz6XMb4MgE2OjHm9mvzTRkHJmKcVSOuz+iGXlqvMM1uYbm9rRvV2ezDTIElXrI8IHgNSUtWHJy1JIJKiJRgjFE3muECjSSvHaMDwuEVPiiHYunUrTz/9NJZlcdlllzFv3ryc159//nmeeeYZ/H4/AJ/5zGe47LLLnNd++ctfAvD5z3+eSy655GQM6ZTh6L7YzWTs1dmEUuemlRvuVkViLlcmcGel1H+2Lsw4y+k+lq2uEILlF9dQUuGnLxQY8rjTPZsoR4hOSnhyvXphRhNct0y5I90eRFbhmF3d7vTDtjuX2UQj6hotJFWhOeXELTCEcty5BJT5TJoqvVT4XDSlXaANFR7ue6GDtlAUiaCqyDVsbtETNgSWZfHUU0+xcuVKqqqqWLZsGXPmzKGmpibnuAsuuIDvfOc7Oc/19fXxi1/8grVr1wKwdOlS5syZQ0lJyYkO65QhEnGkHZwrLVc6L/v2KCOw8y018W9Pr9hmfUCll1qWiiMIAXVNahU3jnK6T2YG0HjKJpJp9yF1jYjVG9UixG5cb6WQwkDWz8RYuDK3H7ZsziQkWClV3T7Qp3axe1r0DnQE8JqCmX4fLcGYEyPwmoJYSrLsounEUpL7X9zPloP9SAmVPoP1n22g3GuOjRhBc3MzU6ZMYfLkyYCa8F999dVBhqAQW7du5bzzznMm/vPOO4+tW7fyiU984kSHderI3rbXNWWpQhbQH1qoJnvL5VaZRHvanAXZeMrpzvb924+zt7r2Ct9jwJoX9hfMrbaPkfLIn3XakF/JDqoi3Zabttm+VUlSeH3qX/t6rJ+JuOZmZTz6euCfH3XkzR3X5ThYhIwW7N2uvZMF0okR7TSUu1l84TTagzGns2hjpW/YjACcBEMQCASoqqpyHldVVbFr165Bx/31r39lx44dTJ06lSuuuILq6upB7/X7/QQCgUHvBdi0aRObNm0CYO3atVRXV5/o0I+Ky+XK+R4pJcSiyNvWI+IxpJR0LfwKMhwAAe7z5pBq34Wr8SzK/P70ZF8FPSG69u9B9gQxOtqpLi1B+IqGdewjiZSSc6Z0sfNQH2dNKmH65Ik5E/zS53aw82AfM/xFtGVN8uaEciqLlMLm0ud2sPNQH2dOnMDZk0t593D/oM8aDZys8y6lJHTGOSRbd+JqPIvyqioO72sreKy/spLe768m0bxdNa1PJjD2m1RVVRFafweJlh2YM2ZSueHfEL4iwmuXkWx9B1fjWVQsu9c5f6PpmjlextrYI4kUraFWuvoTSCmpqqrinKnd7DzUx8zqCdz39+cU1ow6RQxLsPhDH/oQF154IW63mz/96U9s2LCB22+//bg+Y+7cucydO9d53NXVdbKHOYjq6mrnewqJzkkpkYl0jCAUJBGPw+2PEH/8Prq++T+VaS+aoLbo0oJyP1ZNA129fYi+/mEb+2jgpo9NJJaqxmsKurvVKldKSTiWYvv+EIGoxeH+OKaA6mI3iUSKbz7zGjP9PhZfOI3tnWECkVRWNtHEnM8aLZzM8y6vvhniMZIeL129fbk1AuDIUHffuwx2t2Yq011urMk1HN63F7a8AqkkqWA3Xffcgrh2KXLXdgh1E09ZdO3vcAQOR9s1czyM9rHnx7UsyyKZtFT7kaRFXyjAko9W0xP3U+oW7D/UdUpiYNOmTSv4/AkbAr/fn3Mzdnd3O0Fhm9LSUufnyy67jJ/+9KfOe7dv3+68FggEmDVr1okO6ZQgY1Hl7w8FAKmKemJR5XtVRygj8fj9sG1L5o196bTSiqpx3T9WCDHIHXTP5g6aAxESVsatlpKw7vJzuPJnb5ICtnQOsOb5ffTGUrgNQWOld9xkE+W4D+MxFZuyMUwlQhcOKpdjzQzYEcoUlu18C5Zfk0lmANj5ttrVjrNEhZGmUFwrkkgRiqmswlAsRX8swfdfOURLIEpKSkwhaPD7WHnx9GHZGZywIWhqaqKzs5NDhw7h9/t5+eWXueGGG3KOCQaDVFZWAvDaa6858YPZs2fzs5/9jL4+NZm+8cYbfPWrXz3RIZ10cuQkhFD/Lr9GNZbJvtFqZsC+9tw3TyhTO4KaeigpGxcT2LEQTVps6ewnYamUOruJW6kbfvDyHuwuDkkJbx6OAlDhhSUXTsvt3nWaZxA52G1Pt6cXGWefp66r7VuVmKFhwrkfVEahvzdzXRpmpro9/ZydkaQZHgrFyHrjuR3n7n6hg+ZAnETW04H9/az+z73c9qm60V9HYJomV155JXfffTeWZXHppZdSW1vLz3/+c5qampgzZw5/+MMfeO211zBNk5KSEubPnw9ASUkJX/jCF1i2bBkAX/ziF0dnxpAduLNTQZMJ6MvSc7HlI66/HW75dub58kr43gbVh3hvG3LjPchxXjyWHfS1A+z2fsBtCFwGvL5P5b8LVOck2yhIRE4q3XjKIBJCYCzMTOAiHRyWK69T2UB722DVg+D2wA/vVa1PIbcxkulSOwuvz8lI0gWNpx67PgYyVfITi12YZK7t/eE4yQLdSN86FCGatChym4NfPImclBjBBz/4QT74wQ/mPPflL3/Z+fmrX/3qkCv9T33qU3zqU586GcM4JeTISeQXkYHy055zXkYMLDtPu78Pnnwgow4J4zp1L7/M/vzJRTQHVM60aQh6YymCMfCYAn+RSUOlj1QqxdaDavJzGYK4Bb70Tvlo2UinG7YQnYPXp4rI2ptVDGr1onTWUN45MMx0DYGEuxYj7SK1cEC5ksbxNTkc5GcMCSEwDIN//kIj617pouVwD/WVRWw7FCEp05OyQdowjJGsodOZnABxXaPaere3KL+/zfceRVRPVnLBj92rOkbZJBNqqz6jUT0e5z7Z/In70b+r54GXOmntjlBb6sZd6aUtFOesSSVc88EqyjwGa/6yH7ehHEeNecU1hVZapyuFROOcwrPeMHL1IrXYEGKwwu3Z58GVNypDEepWFe92UaS0kG6PLjM7xWTHyLLjY2dMLKGpqpjdoRjFboEh1LVsmCYtgShNfh8+V6bZ0Klyg2pDkId9w0m3BysUgPZdKiAHcOcGePy+jLIowL/8AOlyZ7qQ5VPbgFi4SukMjfO87fyJWwhBayBCMGYRPBTlA1OKeOB/1PHkliCL/9BOXbmX3aEoCUviLzJZ8onpOeev0ErrdORIbVIdeYkZTaqITKa1rezdq+mCf7pFTfp2M5veUObDDdPRwNKcGvIn8FhK0hyIEIik+O89YUyhYmFuA0q8JoZpsvyi6cQtnPecajeoNgRZ5HQcsyy6ogOQSKibqS6daZEfDLa1XbKxV2UuN/zTLSrqr2+0QRO3lJKshCHeOhhh0R920x9PkbAgGBmg1GNQ6TOYmbUyyv9Mn0t9VjRpnZ4G4Sj9hoUQiPnLkY/epYrGaurV7nNPi7pun1qv3h/pH7xYmVY3rnepp5pCE7jXFDRUeAlEBpBkjICUEIykaA3GiKVyu5adajeoNgTZ2DdcOJC7xZ5QCt9ZpHYDvT25mRj5FJWAy1TVnEUTdBP6PLK3yJFECiEzEbKkhFA05bgpJNATtzh/chHLLpo+5AR/2geNj6HfsEjElUJp+toVqzc6K0m5ar56Pl13kIN5aoOQ450cATlpT+AGKy+p4Xv/uY83Dw5gSZjgNmmo9LCnJ0FjpZd1L+6nNRTLMR6n0g2qDUE29g0npdpGRwfUzzOaVNB325Z0E/pSiEVyU0dBqY/e8zjcvlAdZxh62z0ElmVx3bNthOKDX8t3sG0/HCGWkhQNkU59ugeNj6XfsHR7oLZePbCNRdodSV2juh4nTYFd23PfuLdNuUI9XmQ0Mir6555OOKqjQv3rSV/DhmFw2yXT+c5v2ghGkoRiKVoCUR67vAkhBAuea8u7no1T6gbVhiCL7BtOuj1Ul5ZwuKtL3YB3Lc7sEAZ6VaMZgEhfRm7a5c50Hkv7c/W2uzA9cYtwbIhdVR5JSxmOaLJwb4LxEDQ+kjaV03hmTxvUNsC1S5GPrFY7hLpGFTfoDQ3Oeiv3Q10TlmXBhrvp2teGVdOg00lPIo7qqFT/xlISYSkXZkIKRNaypzcunUVMY4UHyL2e84syTybaEOQhhEB6vAjbD/vUemh7V+0QsjEMxO3fR3qL4OZvKf9rbwieXJdRFtUMSZnHoNxrEo6mcnYAtqMi20SYAu79y352h6I0+otYefF0EjJjFMZL0HhIsl2aoBRKt6d3r9t7Bu9cQRU6rlwPP34YVl4HA31YyYRqpKTTSU8aXlPQ5PdBOgMo2+Wz7KLpnD25lJfbQwCUegQb/nqA1kCM+kovj/5dPUXuMaI+erqRnaERbjwT3vpbJh5QUgYD/c6qX04ozRgB9WZH5lcX7BwZwzD40bxGDvUnuOWPu+lJKHOQAqqKXEwpcbEjXVFc5BK8cVD16X29c4Bv/6oVlyFo8mfiAadytTTqyW9ks6+9cAZbNrX18JOHYVs6A850YfirsWp145rj5ahpnek/RcqStIdiBCJqPomlJHd85ixu/fWbtIdi1Fd4aQ1E6Y6kCESTPPDiflZdWjssv4M2BPlkZWgk2/NUVE0X3PejzI3SfSjTBQqUa6hupvpZdyA7IlJK1v6lk+ZAhIGkulPshh1SCPb3xpk9pYjrPz6VJX/cDYnMHqEnlt5FBE+/eMB7Id+lKTfeo15IxJXcRO7BgIDm7bnBmKIJ+B/5GYH+Ab1oOQ6OlqgQS0la05O/IE5DhRch4jRWeFj/UietoXZCAwmSEgjFqK8sIhBVPQnaQrFhu761Icgna3XlajqbeCQC77ypXpsxE1FarjqQbd+au+UuKYO7H8MoKgZQed2g4wRDYAd4AxElJuf3GTRWeLj6w1NYvmkvgaiF0ZOgyG3S5PcR7Oh35q1yn4kpxGkbD3iv2DUwYv5yZG8Yll41+CCRblZj6xk4qc4u5Q7VHBdHS1TIj18tS9cHSClZ8Gwr3ZHMAseSsOKTU1jzl07aQjFm+n3Ddn1rQ5BHdpvA8qoqDvf0Oq0BEULdYNm6Q6BurtU/wEin4h1Llsd4J/sGaaz0sviCqTz48gGW/fseLCGo8KqAmc9lsOTCaVz32xaCUYtKn8HGv2/AMIzxGQ8ogJQS69G7MjUtsz4AVy0ufHB+2vNZ74fOfWCl6FryLWRto3ZlHgdHS1Sw41fRPCEhj8EgbaGemMWav3Sy8pKanGKy4UAbgiyyq4rlk+vo2tuqNODtuoH+vsK+V2nBquuwTBfUz8zcSNodNCT5AV6n2jKqJioTSKQspFSpc2dUFdESVEG24QqgjRnyRRH3NKukh1kfgN27lFpuoYBxSZnarUoJ/X3IVFItS7Ur85gZKlEhO24AsP6lTkdiWiCpK/MgZK5RlkBrIEpP3BrW7mSgDYFDThl/bT3saUt3HktvnQtpuDjiyShfrFQ9CWQsqo7Xu4Ejkh3g9Zo41ZaggsZbD0a54897ueNTteM7K+ho2O5Mu1d23UzVF3vhCqzIAKz4p7xm9gJWbwRfkUqL7gmCy62Dxe+RoXpt2HGDxRdOU+6jSMYYB6OFZcAtBIt/356TCDEcaENgk1/GX9uAMA2kvSPoDeeuqkorVL0AEva2Z5qE1zYgH7tP5XBn7w40R0QIwcpLarj9P/bw5qHMTfLGwQirn9/HbZfWOjsHr4k+p1koieqVORLVdlUxT67LBIwNM7OgWXOzSn5AqnqC2nr8y+6lu6/fKTDT5/j4sbvu2VpC9nMzyj1IyyKRsuhNDP1+w0AZjGFOhNCGwCavjF/MX55bUHbHDbmqo6BK+ResVOX9bo+6gR6/T6XkSak0i/Q2+7hoL1Bq3BaMEoom2fjXg6evjMQJMkiiGtS1t7cts6M9+1zY3Qb9PZnOeqYLznwf7G0n/OAdyEQCWUDcTnN0nJ1AIIplSSp9BnWlJqv/vId3A3GEgPdPLuaNzgFSBd5f5oaZfh+taRfocCZCaEOQJj/AK6Uk9MBt0PKOqs6sa0wH4ySQVnBMu4GEkW6dKARyb3vmxqtt0NvsPIbKuZZScnggOahzk4FqSLPo9230xaVqayl12ugx4fEq0TnbLSQpHOPq2AM9QZKtO1XhZChdmJZexBSSwNYMxskgiiRxG4Jil2TLwaxMLAntoRjnTy2mNRilL2Y5gnMTPCZnVPkGqY4OF+PeEORf5NLjVZ2fHruPxLYtKhC8vUcJz519LnxtPvzbE6onbH8vrLgWK11gJuYvz+wqauuV/LS+cZzJ32PAmhf2O41pllw4zVEUvWdzB+929Q/SGZo10UtHb4Jg1HJaWmZrtmiGRgiBuHoJcsW1Kg7wzpt5hkDArNlq0bKnFVfDmSQiEfV8Ou35SBLYmlzsDCIpJb2xFOECOloNFT5u+sRUpJRc92wb4ZhFwlJii83daU2tU9yNrBDj2hDkX+Ri/nJVjLO7WWVa2MqYlqVupJ4gfO8G1ejDbhxuN6ff3YJIxDPyEnr1BOQGzurLPbSH4nRHkvREU1z32xYaKrxc//GpNAcihGKDV6v7exM0VvpoC8XoiaZU4Y20jihCp8kgvD5kXQNsC+UaAWHAvU9hVPgBsKIREquuU9d0SRlcuUgddxQJbE2G7FTRdS920NwdJRjL7HA/MKUIgWTh79qpr/Ai82RrQlGLNZv3cfulNZjDrAp7UgzB1q1befrpp7Esi8suu4x58+blvP7cc8/xH//xH5imSVlZGddddx0TJ04EVEvLuro6AKqrq7n11ltPxpCOjbyLXNUINKutscsNlVUqfTTSn0n6TSaU37W2EWjLBInTKyidNppLdsGNlJKGSh8SSTiSIhiVBA9EuOY3bRS7Ve2ARBBJShIpicuARr+PFemb69rfthKOWYRjkgf+so+Vl576pt6nBYaZDgwDRcWoFX8jorwy0+AmEUf2pGNgfT1w2wKs/F2uLo48KkIIitwmKy6pJZaSuLDoiqQocas2q0v+0E4gaiEti4G8jF6JSo749q9a+PHnZ6o+JsPECRsCy7J46qmnWLlyJVVVVSxbtow5c+ZQU1PjHFNfX8/atWvxer38+7//Oz/96U9ZtEitODweD/fff/+JDuO9kR0grmuCnzyidgIuN8yajf/amwnc8DVlDITIZF3MmAnzl8GGu9NGoR6uW6azLQpQqLIymrS49jcthONqhZqwJD0xyflTirntkumUVPi59Vdv0hZSQTdQ2kSmkc7RBlpDcR0nOBbiMdWsJpmAsgpYvg6eeRT27UZuvAe5YAWQlrEuK1c7AiGUgJ3e5b5nMimlBpNLTO5+fh9bO/tJS2oRT6r4gHM8GcWPcMyiJ25R4RtDhqC5uZkpU6YwefJkAC644AJeffXVHENw7rnnOj+fccYZ/OUvfznRrz0p5Gi02A08kgkoqwTLIrjiOmUEQG2rz3wfXLcMw1cE8RjW3rbMlvnRu7D2tWs/ah6FG3eDaRhka4xKYE84pqR5haA9HCMUk2w5GOGuzR2svHg6TX4fvZ0DgNTyEseAlFJJTNfMUOnPPWFY+p3sA1Q71mcehb3teM6YRfzLV8MzG5Tx0LvcE0ZKSSia5N3DGSMA0Je3Gzhvkpe3DsWwAJcBJS6GtePeCRuCQCBAVVWV87iqqopdu3YNefyf//xnZs+e7TxOJBIsXboU0zS5/PLL+chHPnKiQzounItcyow+UFrBUdqTvM3+vWpl5CvK3U2kC9AcGWDtR80hv+DGawpmVvloDkSZUeZGGAbtwRgNlT7cQvK9P75Lb9q3KiW0B1W15fKLphNLqbvJ5zK0sT0CTvzLbqaUSjGo5U84ALd823mYbHtXaWUtXKl3ACdAdnLEPZs72Hqgn4R15Pfs6UlQ5hWEYpISt+Cev3TSntWh7FT/HYY1WPzCCy/Q2trKHXfc4Ty3ceNG/H4/Bw8e5M4776Suro4pU6YMeu+mTZvYtGkTAGvXrqW6uvqkj0/e/iDEolhuD93f+YdBGSyEA7D0KszZH6Fy+X2QPl56vITXLiXZuhNX41lUTBu6reJw4XK5Tsk5Olms/0K1s+KJJFLc9oedNHf1c/8rXTR39TvbZrcpsITBkj/u5uzJpaz5u7Odjk1Hal0ZTVojYixGy3mX0Qhde1tVjwGbgtXxGVz1M0fFtfteGKnznn+tSSlZ+twOdh7qY2b1BFpDsUFGwBSQyvozGAIQJudMLqa5O8LM6gk0d/XTPZDEMAxKKvynPJPohA2B3++nuzuzcu7u7sbv9w867s033+RXv/oVd9xxB263O+f9AJMnT2bWrFm0t7cXNARz585l7ty5zuOurq4THfqQWNEupDXEDZNMkPjbf3Go+V2MCn/6pulHXn0zxGMkPd6c8zFSVFdXn9JzdDKQUrJ8cwfNgQi9MdWw3rJ6mDmxlK7+OFLCBBdKEC1qsaOzhxt/8Qbt4aFXSiPdv3i0nHcpJVZNo3IH2cagrBJuvAPu/G5BuZTSRd8bFdfue2Ekznuhay2Wkuw40EP3QJJkMkmqwE6g1A21FcoVBEreKRRJEIsneOSzM/CagjUvxLEsi4ZyN32hAP0n6RqeNm1awedPOBrR1NREZ2cnhw4dIplM8vLLLzNnzpycY9ra2njiiSe45ZZbKC8vd57v6+sjkVAXaU9PDzt37syJLYwEUkrVVCbSD243NJ1d4CALbr0S69G7VB0C6ZztdGm/5tjIlqIGgb/IZKbfxwP/cA4fnFKMv8jkjOpiZvp9VBW7qK/00hZSGUgtwZjjJir0mUc6ZjygZCdWwPpn4NwPQkWVSnLwT1RtVoWAktJMNpHLNShLRUqpamqO1uRmnFLoWrOTI6qKXTRW+nDnxbHcBpxRXcyCj07NeV6iis1UDE31J974943DtpA54R2BaZpceeWV3H333ViWxaWXXkptbS0///nPaWpqYs6cOfz0pz8lGo2yfv16IJMm2tHRweOPP45hGFiWxbx580bcEOQoObrdcOhA4eOkVKmmOh7wnsmXorYLzEzTZNWltTnqjZmCtA5HhbRQsHg89C8+VoQQmEXFyBtuVxP6Ew/A7QuVRtYV16t6gY1rYE+zypoDp3m9LiQbTH5VfKFrLTs5wi0k3/5Vq/N+ATz2Dw34iz1YloUrS4q60ity+g8Md8c9Iceoud+/f/8p+VzLspA3fyvTeUxKlU6KVHu4bD330nLEAz8Z1nzfY2W0uCiORiHJiaqqKjoOHs4xAvbrR20LOMRnDhej9bzLWBRr5XUqy62iCrF6o3revv2fXIfR0e40ryceyxxfXgmrf4CZbro0GjnV530ol+ORJFPCsRTffa4lp1DyQ9MmsOoS9d5kMsmAq4SiRC8pYQ7L9TqUa2hcVxbbZMtMEI/l9h0oq1Tpd3vblb5QNgP9mSyi4/iO8b6yyiZ/5WNZFjf9ZhvvHuyl0e9FIGgNxWis8LD4wmnOSuxIKqTjun/xUORkuSmFXKfz3lnvh47dWKFup3m9dHtgWp0yBOGgklJ54MejctEzHAzViayQBLWqLN5PazAGwsAk5SRKtwWjTnD53hcP0BaO01DuznEBjcRCZtwbAqe7U3oLzFVLMi+63DC9ThkBW63xvX6H3mYfFSklq/9zH68fUD0Jeg8MMMGtUupCA0m++X+bASh2GxgCzqgq0iqkx4G4aolakDx+f0YhF2DHG3DWuQgksrYBy+WGR1bDu29n3twbUhXHZRUjMfQR50gux3wtrezkh6oiF0/+r0YeeeUAbx1Uz697sYMln5juGJZUKkU4lqLcqzKDRiLZQRuCWFSpiiYTqrFHKsv1Iy1lBPrC6abfeZx1LuJY4gNar+WYiKUkbaFML4KkBeGYxESVntkZGOF0jUFzIOrsDHTTmlxydrmQ13SpNTdryErBjjdUunRvDzy6Gt5+PfcDS8tVTGGccqROZPbEPaPMTXs47vQhABX7qvC5kFZaORdoDSo1usYKDwDJlOU0o3Ga2AzRA/lUMS4NQf5N4pBMwGNrMg1o7CKc0gqI9KnH9g3kcsO1S49t4snrdaD1WgrjNVXAbOuBAZJWpvypkHY7QE2pi1Qqxd1/OUDrMBbfjHYGiSletSSv6dIM2Ccy13d297JUEtpbMo9dblVRf8Pt49YtZFPI5ZjtMgpHkioinP16MkUomuTNgxHnuXpnR6G0tQYSFgnLgqB6fSSSHcadISikOMo556kVkJTQsU+tfOybIxqB722A+5aqgjLDhNIyJToXj5HyeDGSiSP6/nUz+6Njb6+XX1yDOaGcb/zzqwVlfLPZ0RXnil+2ICVOAZrWH2KwmKJMJzkIoXa3e1JqQbJgBWy8G3a8mak1ADWZlZSrHbFhgqGruIciV3raclb9Nju6oiz+427scgKXgJsunEokkeK1/RnZdb/PoKnSi89ljEhb1nFnCPJvEpGIw8JVyEfuVDIR9TOVBO9NV2RuDrdbNabZg/r3qiWw6jp1jMuFVVJ+VN+/1msZmuztdWOFh7v+vgILAzhyXX72TecSaqs9ntNFHfK77QmBFIZa6CSTyt+/t03109jbrq7z8kp1fDioJv9ptZn+BW+/jnxkNfKG27RByCNHevolFSBurPAST1nsCcXoS0iCaVeRCcyeWozPZXDHn/fmKBes+fQMJpd4nPM73IuZ8WcICrhpBCBNl13rrbKAfEXQl1A3yfJroGiCel0YypjY6aXJpPb9nyDZ2+ueaJKvPvM6kaOJs+RR4hEs+NjUox84DsjfgQLI+pmqdWoqCZEBZQR+8qha2CChrgm3yyTRusvR2sqJI+xt09f3ECiXkcGSC1Vqpt1sKZq0uP/F/fxtfz+gljUSlfG2N2u7W+YRTJrgHlEjO+4MQSE3jYxF07uEANCC6O9FGlnaHslEpgHNnhb48fczDmzTpQJp2vf/nsnfXncPJDmSsoqJavJt2woBIAyW/GE3TX4dJ4DBO1Axfzny0btUoFiIdF+NVpg+Q13Lbe+SMAz1eMEK+OFa9UZbgl1f30MyVI1Bkdtk5cXTueM/9vDmoag6zUFVqT2zykdzd5SZk0pZesHEEY+/jDtDAAXcNDl9CRqRLndu4Rio7XJJWUZpFKlqDFZvxDBN7fs/AfK3122hOMH+RMFjyzwGGz83gwW/20M4pv5GArVZ644kIahWYna9gf6bKEQijtybVsi1z0kqqVb6PcHMgeGgihssWKniZP/8qFr85EdBNQ75zZeiSStHJM40DKeKuDeWYv1L+1l+cQ1xC6ZPnjgq9J3GpSHIx94lWJEBePxeWDVfBYmzkRJWPYgoLU+3s0zXBBQV68nmJOB0drq4hoSnhC/9+G/Oa+dUe9nRpQS6TANSwsQQuUG5hgofQsRorPSqYh6dRZSLs9iRqvlSMqEKJw1DZQZlB4v3tIG9e+jvTWfRtehm9kPgNQWNlV56oimnTmDFJbUA3LW5gy0HBhwHQsJSTZXi1uiSUteGII2UElZem/H9m67cGyQtKmcYhurqpG+EU4IQgimlXsp9JuFoCtOA/eEYZR4DU0gaK32UuJSv1cYQIJE8+nf1CCFY8FzbsOdhj3bsxY6jObS7RU3yyURGeM5mWq163a6kd6WTJXQz+4IIIVhy4TTmP9tKIJKiORgjkkhhGAbtwWhOUNhkdCY1jO/E4Gz6enLzqc84B1GaLqARAs45zyke00qjpxbDMHj6fzXx2OWNlLoNwgnoiVukpKA9FOOuFzpVamOapIQtBwZ44MX9OeqP4110Lh8hBIavCGPhSsRdP4BZH4AK/+BiyV3b1fm1n59QCt9cqH7Ozrrbnd4ljFNsN5CUqj9Gk9+HAIKRFNf+thUXFskCcvbzPzrFeX8kkcpRd83+zOFE7wjSyOKS3CwJw4V75jnEd+1QcYEFK3VP4mHEMAwmTXDTWFXE6539SAm9cQsJBCL9lHoMKr0CC0FPLIWU0BqMOrUIutJ4aIQQCF8RcmF6h/DDe5XkhE0yAUaJ0iDq3KeMwupFupl9Fk6AOBClvtLLyouns+CjU/jWL1UxXjimmtaLvPZWKWDxH9qZ6fchhEFbuM3RGoKRkZcAbQgcxEBf7p9sXxtlDz1Dd/+A2hJvvAervVkZhWtuwfAV6UnmFCOEYOXF07nr+X20BqNYUt1gErVDOG+SF7dh8MbBCEkJfXHL8c8eTZjudCTfd1/Il28/J90e1Zje60Nevwoeug3eeSvzYVYK3t2mfhZCxQl0M3uHWErSEojSHUkSiCZZ/fw+brpwKmUeQU9cOS6feO0gDZU+3kpfn650g7hg1KIlqHZSgUgKy7KcvhkjIS8B49gQDLpJSsqUlITtF5WSwM1XImsaVIl+e7PKuAgHYPE3sGbNxli4Uh07zm+KU0G2AuOqS2uJJi3ue2Evrx/IaBFtOxSjzGc4VcUJC1qCMULRJBv/enDEupSNBIUq5nOSGhasANKaQ+3NYKVUinRdk0oI2r837xNFJnPO5VYuJN3M3sFrCuorvQSiSaSENw8MMP/ZVpoqfbx1OELSgr/tH6DcKzhvchE3fUIp565/aT+tobjSGRICM5ygodztuDBHqpfGuDQEhW4akYgj7/uREtxKq41aveGM0lltg0qzk1JtnXe3qG31k+t04Owkk5+XvSzdtL41ONgfXV/hhVAMS6qAlyVh8e/b6Y1nyv3HRcA4X9iwrydXZiIWzRwTDmTet6MXfMWqTsYmXSnPzvQO4ZzZiGtu0nGxLOzd6urn9/HmgQGSEkIxyZYsTSGJeq49HM/pPBa3cCb5kgo/faGAc15Hyq05Lg3BIC2WR1Yj97WrisqOPWrCd7kx/NVY02coP+jClfDI9+Cdt9VWuaZe7Sq0quhJJ5JI0RyIEIioQNpdmztoD0bpjQ8OoAkhqK/wsjscp67cy55QjEA0hdsAf5E5fgLG+RXzJWVZtTFNmUwhaan6l95QZlGTbQTKKuDOjfD0QzChBGobEQtXjHjB02ghe6dqGAa3fHI61/22hWB0cCX8BBNiFvREU1z7m1YMA2b6fTmTfZHbzOlHPFK9NManIfB40635ZKZALBxQaow19eoGqW3A9HiwWt5Bbrhbva9jL5xzflqcbjc8uc5p8TeeA2cnEykld/xxJ72xFG5DTfLtwRjdkdSgkqYU8HrngPO4NxZh1kQfQiRorPSw5BPTR1Wu9qmkYMV8+rGUErlqvrrGy/2w6kH48cO5PQlMl6opGOiHp9al74kgiHalx6UXOAUriH0ug5lVRWztHEBK6bgpAe77H7Xc+Me9JCwIx5Wh2NI5oBY26TqX9V+oHqHfJpdxZQiyA2Ug0zIRhsqR3tGrVkk7tqZbU0LizdfSfQq2QnFJ2jVkAcLZXovVG9VEo2MEJ4VYSvLu4X4SFviLDG7+xDQefLkTApCSkp5oakgpuoQl2R2K0uRXDWvG2yp2kKyE/VhKZNZuwSirwFq4Eh6+A7a/qQzAmediHNyHFehSrtHaBi0tkUfhLmUGK9JV8ZZlcd2zbfTELEwByzftdXoS20gJrYGIs4OI5h8wQpwUQ7B161aefvppLMvisssuY968eTmvJxIJHn30UVpbWyktLeXGG29k0qRJAPzqV7/iz3/+M4Zh8O1vf5vZs2efjCENIicukL0L2Ctg5XpYvSjjO7V1WDJvzuRVWxLqm9T7ZjRpv+lJxmsKzppUgmX10FTppchtOltpjwHBSILv/LrNyfASkJPtZftk4xb48uzASPYyHkkKidDxgzWwbw+YpgoK72vHOHMWliVz4mZHykAab3hNQWOFByklDeVutdOS0qmKjyTs4kZV2xKOq0whIVQig0DZXIlw3JY+l0H/SP9inARDYFkWTz31FCtXrqSqqoply5YxZ84campqnGP+/Oc/M2HCBB555BFeeukl/uVf/oVFixaxb98+Xn75ZdavX08wGGT16tV8//vfPzUrufxgWtaKR5SWI+uaYFtY3RQuN8yYicvtJtn8jupZvPNtZRAi/XD1TWqM4/imOFUIIVj7uXOc5vX5srzFHhemUDeaCZwzqYhthyJI1OPyIeICQwmDjReydwtWNAK7m3M1hvrCVCxYRmAgMigzyFlEtTdDbQPi+lXjYrdlF3dBRlFUCPVvczDO/GdbHZ+/lJJ7X9hHKL3SN4FSr6ChwsvunoTTtSxhgSlg/WcbKPeajpEd6QXKCRuC5uZmpkyZwuTJkwG44IILePXVV3MMwWuvvcb//t//G4CPfexj/OhHP0JKyauvvsoFF1yA2+1m0qRJTJkyhebmZs4888wTHdZg8jXas1Y8ANIQSkU0q07AX11N1/4OrFQKlnzT+Si7slhzarBlfQvhcxnMnlJESyBGSko6wjHKvAZ9MQshoL7cw7KLpg+6oYZqPj7ekFKqwHFfrxJSFOk00dJyRHmlUkbLL5yMxzLp0z1B5KOrkdef3r0JpJTc/fw+th7oBwSzpxazJN1GMpCOV0lUuvJAPMmaF/bz1qFMxpBhgDAMTFNVHNMdoSdmkZKQkko80TYCo2GBcsKGIBAIUFVV5Tyuqqpi165dQx5jmibFxcX09vYSCAQ444wznOP8fj+BQIBCbNq0iU2bNgGwdu1aqquPP8gib38QYlHIc+fIaITDe1uR4QDCNJg4cSLCV4TL5aJ62nSC99xKAsDlxj37I1ROH/2rSZfL9Z7O0WjgSGOXUuLzHcZwJemLJkmkLPxFJmU+g2DUYk9vkpIKP8Uel3N8NGnhNwXnTOli56E+zppUwvTJE0/J33C0n3cZjdDV0Y6VTCD81VR+/18w+nuhejJutxvXEw+QbH0HV+NZVCy715msgk1nkdjyV5AWxv49VJeWIHxFI/3rOJzs8x5JpGgLt6alziVtoThVVVWcPbmLV/eESKQkbgPOmlTCgt/tJhhJZsZiqMVMMJJit5nkX77+AVb+fif/vScEKPeRq6SCyiI3SQzawnG6B5IYhkFJhT9HuXS4GDPB4rlz5zJ37lzncVdX13v/sL5cr5xlWci07rpMJjnc04vR10+1vSNo3pHu4uQnecUNo0I29mhUV1ef2DkaQYYau5SScCzFjgM9dEeS2Av6QCSFSyh/bDiSZOlv3mLlxWpXkN35bPGF0xCiCp/LoLu7O2dLDpyU7floP+9SSqyaBkhZyJoGAg+scupgJt56D/Fd2yHUTTxl0bW/Q1UeS4n1re9C/D7YtxurpoGu3j5E32jwbitO9nm34wDhSAIQNFR46A8Huf7D1cw/0EMgkqLUa3LFeZVc+9uQ875ZVW6WfGI6P3ztMK2BGHVlbrq7u2k53AeoOEHKknzrp6/T5Pey7vPn01DuxrIsGsrd9IUCOemkJ5tp06YVfP6EDYHf78+ZGLu7u/H7/QWPqaqqIpVKMTAwQGlp6aD3BgKBQe8dDkQirqospQTDzE2Xy3cpaZfQiGBvoZsDESwJbkPkCHOlJJR5DcIxi9c7+7nr+X3c/MnpTj1COJJkwXNtOT7dbCMhhDHi2/PhIDtw7KSVhrIWNnk6QjlJFnVNKkvuNE+QyO6fHU0q+Ydyb2aVrgLGMWaUuZngglKPoDcuMQW8G0hw7bPtnDe5iLoyNy3dA9z3wj4a/V4IQF2Fl93BqNM7Y7RoY52wIWhqaqKzs5NDhw7h9/t5+eWXueGGG3KO+dCHPsTzzz/PmWeeySuvvML73vc+hBDMmTOHhx9+mM997nMEg0E6OzuZOXPmiQ7p+CnQvtJGN54fHdg+/kAkRYVXMMENoXShsd2Ypi+mAnVSQltITXRWuro4KdXOoSUQG6TrYhsUO6B3uscPctJK6xrVCatrVC7Tq5aoY9KTfaZ7X7fz3tP5HshZIFR6QUJrKOb83ByIIKWgL26x5WCUb/yyDZehembs7I6RkoCElkCUvriqK9hyMMoHpxaz4e8b8JqCNS900BKMZbKGRqiILJsTNgSmaXLllVdy9913Y1kWl156KbW1tfz85z+nqamJOXPm8KlPfYpHH32U66+/npKSEm688UYAamtr+fjHP87ixYsxDIPvfOc7I5aNIK5a4qSCDXpNa6uMOHbqXk80SX8Cit0CV7o5jUwH4GxMlFaLKuvP/XvWZ2UU2boutu6LSN+c46IS2UYIp/lYaM1SZHNabXfhKqc+ZjypjeZ3G4O0MJyUhCODa1gkKhNoe1eu/EkylVtc1h6K5chMjPQOIB8hh1v4+iSxf//+k/I5OVvfVFI1p6+fibFwJRMnThzV/t4jMdp91UdiqLFHEimn+Ye/yFRBzAKl/ZU+gx/8QxM+l8Hdm/fxWkc/EtUk/Mefn4lpqm3+eIwRZCNjUayV16nVfoUfYRjIQJcyDu/7IMYNt42ZGoKTdd7VjmAfLUG1C5BS8saBCElL5tSruNIpzENR4oJIUlW/A3xo2gRWXTLY5Tjc18spixGMefLrCwC2b1VbYs2owucymOn3OTcpUgnRhWMpp4LTAGZWFTnppws+OoVFv2sjGLNwmQYJKbC9vfm6LiO9PR92cnp1N+FyGSSC3WqLtbfN0c4aTztiu3+2XcAYjqVY9Ps2gtHMrH/eRDc3XqjiTI++cpA3D0YQaYlpe+KPWYISr0CguurZyQujFW0I7JtByoy66BEYC6uj05Xsm9RexUeTFgPxJEv+sJtgzKKiyGTJhWrVYweXpRD4fSYNFV7cQqWTjqZt+UiRH/+qqKri8O3fVUagfuZp7wYaCiEEXjPr+pG5O4DtXQmu/nUbhgHnTy7ip/97JkIIPAYcHkiy7E97VCabYbL+s/VO4dhoZtwbgkG9XPe0QN3MgtlBul/ryJO9ipdSsv6lTpVJBPh9mbL9/OBybWUR7aEYV/66FUNkVCCP9vcbDVWfpxIhBNLjhXjah33DbeN6oWP/vS3LcjLO/EUm6z5Tx41/2OPIR4BSqG8JxhyJCYBJEwQNlT4EMZr83jFhBEAbAiC7dd/KI98E+W4kLTt9yjnSRJw92bsNKPGaTp9du3exlJKUJXnzwAAScipCj5YdNFqqPk8l2Yub0BnnwNU3j9sUabuauDkQIZWyGEiqnUBjhYd/3nrYOc4lMskJfXHJupf2syLdanLNC/tpC0ZpqPAWrHAfrWhDkMVRfaHjLINipDnSRGxZFpFEygno9cYsgpEUrcEY0aSFEIJlF02nJ26x6HdtjhEo85qYQjW08RjHZmhOa1mKrMVNsnXnuF7cRJMWWw/0p6uJFW5DNZv/p9+2O8HiUq/BjAofe8IxAulrLjslORBJIURh4cPRijYEx4GuKRhehpqILcviW79sJhyzKPMIfvgPjTz4XwdoCURpqPCw/qXOnO5mM6t8NAei1Fd4WXLBFNa/fJC2YJR7Nu87YiGZvauA4W8dOGxkLW5cjWeRHOeLm8EhQnuBkHkhGLUQ4ZhyAYnclOOxer1oQ3CcjKcMipGm0EQspeRQf4JwunisJ65W9PZ9mkpZtPbEnOKwuAXLLppOMJLgwZf2M//ZVgaSgoQlkTKKEGLIQrL84PTpaPizFzcV06aPCfmUU4XXFJR4TUJRJVlS5jNp8vuo8Lk4b5KPXd1RJOA2TWb6fSy7aLrTdnKkW02eKNoQaEYthbKE7tncwa7uTFcy+1ZrDanJvyeaQgglQdFYqdw/dz+/j791DmSt6SRuQ9AbsyjxGFQVuYZcwY1U68DhxF7cjKWJ671wtMB/PC0RDSogXFfmYflF0wFoCSboSajrbXaVh0UXTB2UuGB/9lDKuaMZbQg0o5rsmy2SSLGls5+EJRFAuc/kjCq1YrMDw+FIKp3VoYLEsZSkJRAlf8dfbErCCRVbXvc/Z4zJm1dz7BxL4N9rCuorfQSi/UgJu8PK9x9NWoRiatcogbcPRXJ0q4Axn1SgDYFmjKGmdFPAQ5+tp8LncnYOkUSKa3/b6riNdqf1hpJWbvWxACUYj0XKkjz8Uge7e5Nj9ibWHJ0jBf6zG9CsuGgad23u4M0DA/TFLR74i4oj5UYJlDuxJThYtyr/s8cK2hBoxgyqKU2xI9hlGwHA0XGxt/YCqK9U7g7TyL2N3z/Jx76eOKBiDK8fzFSRj8WbWHN0hgr8ZzegkRLOm1JMKpXeVUpoDkTpT6irx20Ifvj3M/jha4dpCShRurEeJLbRhkAzJsiWBs4P0Nm4haQnrlZ2hoDln5yKaRrUl3vYeigjCnbLJ6bywMsHCRzIxBrKPTCjzI3nCB6i07247HRmqMC/vVOwU0a35MSSoLbMwzvd9rUjMQwDmVa0tVOMToekAu0Y1Yx6bP/u/GdbWfNCx5A3W29COppDKakeCyH47oXTc45LYnDbJdMp96otv0tAXwLeOhTlnhc6sCyLaNLK6XeQPYZ7Nu/LeU0zNrBboGZfOx5D1ZTY5P9VpbQ4f0oxfp9Jsdtg8R/aeePggKofCMUd11Chzx4K2xU1mq4hbQg0o55s/262XzafMo9BuTfdZBzY8NcDSKmaitjeHpcAFxY9cYsfzWvkh//QgJWWsU5YkubuCKv/c9+gCf9YxzAab3JNYaSUTiXwUBPhtq4EloT1n63HFIJg1CJhqeuoocKjmvscx996tC4otGtIM+o51sIuwzD44T80Mv+5NqfKOJaSpFKWUyiUlPCNX6pK4wqfycbP1ZP9aX1xyVuHBhxXQSSRIm5BqfvoY5BSsvS5Hew40KMDz6Mcu+1pcyBCMGrhEiCy1EOd44C2YBSPAQ1+H8H9StK8xCOQCBY813Zcf+v8oLUdpB5ptCHQjHqOxwdb5DY5w5aqrvCQSqW49rdtOTe4vQYLR1PEUpJSr0EoZiFQuwK3IfAXGTRUeLjut22E4ynKvSY/mtdI/Aj3bSwl2Xmob0xnj5zu2Du2dS/upzUYI2WpJvS2BEm2b6jULehLqJ4X1z3bRlOFx0lGaPQX0R6MqZaTHPvfOn9R43MZjIbOz9oQaMYEx1rYZRuNaNJi/UudLHiuzQkg51PmNYglrZwdgQAmeAw2fK6BWEry7V+1IKUyGocHkjz52iFaQ7EhJSnOmlSCZfWM2eyR0xnLsrhrcwetwSh9sRQJCyq9amWfTNemZCOyrEJPzKItHCcpwV9kcvMnpvHgy50QPL5ModEaWNaGQHPaYffVbQlGlS4MaqGnKgcypFIW1z3bjmlAhVc4PWYNVJWpijmYhKMpTAOW/vtu+uKW4zYqJEmx9nPn0HHw8Ki6ycczdqaXx4C7nu/gb51q/S0Av88gaUmS6SygfG99XbmHd7rV5G8aSoW0vSfprOQXXTDVaWx/PH/r0Vitrg2B5rQgP7UzewteX+4mnoK3DkVy3tOndvWkLLj3f9Tz5N8O0dIdJSUlN/6ulZl+Hxs/V09v3GLZv+8hkNagqfQZNFZ4jiBJoXMwRgN2YHZX9wAzKny0BTLpwi4D1nx6Brf+v/Yh3787FHd+LvOY3HxRrdOA5p7NHWw90A8IZk8tZsUYjwedkCHo6+vjwQcf5PDhw0ycOJFFixZRUlKSc0x7eztPPPEEkUgEwzD4/Oc/zwUXXADAhg0b2L59O8XFxQAsWLCA+vr6ExmSZhwylHyAvQW3LIvrnm3NeY9A1RqkpJKqMLH49uwqHv/bYd44EEECWw8MsPB37TT5fTT5vRCMYcl0ywMxWGNmLE8EpyPRpMXr+/tISggeGKDMIzBQLqDzJxcxsdiF2v+pCFL+jrE3vVBwG4IGvw+vqYoWo0krq/ZASZiM9XjQCRmCX//617z//e9n3rx5/PrXv+bXv/41X//613OO8Xg8LFy4kKlTpxIIBFi6dCnnn38+EyZMAOAb3/gGH/vYx05kGJpxztHkA9a/1El/Ok7gyuota0lVZdzaFeWq3+4e9LlJC6UtH4ix7n/OAGDJH3bTHUki0n0PsiWvbd0ZzehASpnTYD6SlJQXmTRW+lhxsVIONdKbt0qvwfcuq+WG3w++DqSUtAYirHmhg+UX1+AWktoyD+GI6lXceBrEg05oD/vqq69y8cUXA3DxxRfz6quvDjpm2rRpTJ06FQC/3095eTk9PT0n8rUaTQ62G6iq2JUjV23nayuhOhUHeGJeI+dPKVLNxoEdXVH6h8gEMoUKDCYtyaLft7HhlU4a/ZnvAY6ptkAzfGTXcQghMHNeFQQjKdqDMac6vcnvo9Jn0OD3Mb3UnVOHUuYxqPSqnV4watESjBFJpLjy1628kW5YX+YbG60oj8YJ7QjC4TCVlZUAVFRUEA6Hj3h8c3MzyWSSyZMnO8/97Gc/4xe/+AXnnnsuX/va13C73QXfu2nTJjZt2gTA2rVrqa6uPpGhHxMul2tYvudUMN7Gvv4L1USTllPdGUmkaAu3qRU9yg0kTBfTJ0/koS9O5tbn3uHdQ70E0ul/hUimC816Yikk0BqK87Nvfsi58X0ug3OmdLPzUB9nTSph+uSJ4+68jxZcLhdVVVXc+ux25++x9nPn8NH6w7xzsJemqmKau6OEowksBFMnVmEYBi7XAfriFm8divDAX7v57VUfIRRNsuZPzezq6uOMiSWYhuDdw/2cNamECeWV9MSaAXV9BCIpzHCCkgq/07f4eMc9Gs75UQ3B6tWrCYVCg57/yle+kvPYztQYimAwyCOPPMKCBQsw0vuxr371q1RUVJBMJnnsscf4zW9+wxe/+MWC7587dy5z5851Hnd1dR1t6CdMdXX1sHzPqWC8jt3OyZZS0lDuJpVK0RuzSFiSvmiSr/zkNaepSDjm5+pft5CQKnh4//9Xw0+2drE1S4QuHM1UIIQjSZb/9i0EwkkhVc1JqvGagu7ubqqqqsZs1tBYv2b2HTjEf+8OkrAgvDtI874DJBIJQKWOYqWwJAQjCRb93zdYcuFU/nt3GFtg+pX2IIt++SY3fHwq7x7uVQ2LDvex4XMNCDERrylI9Ycps5vXGFDuNWkod9MXCtD/Hv7ew33Op02bVvD5oxqCVatWDflaeXk5wWCQyspKgsEgZWVlBY8bGBhg7dq1/OM//iNnnnmm87y9m3C73Vx66aU8++yzRxuORnNMZNcTrHtpPy2BKL2xlCMfHLegwufi/CnFtASiNPl91PuLGZxNDpVFppN33hJQRiIYVf4k28WgUhRPvLJYB59PDLuCPGHB4t+305eQJCwJRJlR6SN0cAApoSUQ4b4X9w8qNHzzwACLf9+OhXIL2qmi2Sq3T/+vJnriFqVuQUKK0+JvdUKuoTlz5rB582bmzZvH5s2b+fCHPzzomGQyyQMPPMBFF100KChsGxEpJa+++iq1tbUnMhyNJgchBEVukxW2QXixg9ZQPKcASJC5icOxFHvDsZzPmD3Zxy2fnM6DL3fSks4a6o9buA2lVuoWkns276clqHoi7w7H33Nl8bE0T9EMjd1q0t7FBWPq76S60aVAWsyeXMRbByP0J2BbXjoxpN090RR+n8mDn20oWCNgGAYV6a70x+8MGp2ckCGYN28eDz74IH/+85+d9FGAlpYW/vSnP3Httdfy8ssvs2PHDnp7e3n++eeBTJroww8/7ASOZ8yYwTXXXHNiv41GUwDHIFxSm7PajiatrBaXA3z3uRYQBn6fQU2ZB5dhsDsc4/6XDrDik1MJxCRL/1+7I0PRFopx1+YO2tOfgYQzJ5ciZe97qiw+UvbT6c7J2AmpVpMi3TtA7Q4sC4RQ2UNvHIry/olehBBOl7sKj1Kezc4uEqg00jLPsamJng6ckCEoLS3ltttuG/R8U1MTTU1NAFx00UVcdNFFBd9/++23n8jXazTHRX5Fp51tZFkWoahFOA5gce4kH0svms71v2unO5KiO9LPV/5PC+U+05GrSFiSYFQS7BxwVp2Nfi/3/f057D/U9Z4mtGMV1zvdOBk7IVsFtKHSg0RSX+GlLV1Z7k4LykkJe8Jxsrvc1fuL2XYogpAyp1+xKQRxC3zjpDZQVxZrxi12HCEUTaqgcTqNdNuhKA+81ElNmYfuiHIfJCV0pzOQ8klYSq5gyYXTMAwDn8tw0hiPxyCMVh2aU83RdkKFdgvZzwEsfW4H2zvDWJbEMFRXupl+H62hOPXlbmIpSUc4Rl2Fj/ZgjHDcIilhW1pp1l9ksu4zM9j43wdpTXfAGy+GGLQh0IxzhBBU+FzMnjKBV/ernCMJtAejFFIItjWLbPeB7VFIycykdLwr3Gw9nKG6r2Ufd7oZiSPthAqdSykldz3fQVsoyky/j0UXTOWdgyrLR6QLBkUgxgOfqUNKyaI/7KYnZiGB4MHcuEDCUru5Jr+PyiI3K8ahIQZtCDQahBAs+cQ0rv1NM6GY8h3XlPt46+BAweMlSoQslWUo+uOZrma2zn0gooKWR/L1OxNdQGkcGQJm+n2DjMfpHEg+0k6okH7//S/u5/V0j+Gdh/u59y8d9ESTuA0ocgncpkHKklz1KyUrkt9jIJ8Sj2DJhdOy6kNOj/N6PGhDoNGgisPOqCqiORClocLL8oum8p3ftBGODd4WpJsTUukVBNOvJyXc/+J+PJ4udh3qxbIkFV5x1D7IzkQXSTqrWbtKOXtCOt0Dydnxm+ydT/ZuoaHCQyiSoDUQcdJEw3HYekCt8t2G8vvXlnt568DAEQ2Av8ikttTN3t4EM/2+cS8UqA2BRpNGCJUlYhiCe188gIESJ7vlE1NV03IpWffyAdoCUeorvSz86GQefeUgWw6ohudvHYyQtNTPLqF06ew+yEOpUzoTncTZERTyT4+WQPKpdk8V2vksv7iGSCLFtb9t5dpn24HM+U1k2emEBcGYRIRjShMwbSwMoNQDTX4fFgZ7wzEa/T4WXzDVUapVvxOnzS7reNGGQKMhs+IORFJIGXX6DvQdjmCapiMfsOKiaaze3MFbByP802/beP8kH+dPKWZPOEZvWooC0umIEkD1QT7SCn7xhara02uKIWMEoyGQ/F5jH4XGawfTAWc1HkupzJ/BOx+DWErm7M6KXer0JvI2bAJoqPQhpWRL50C6JanBg3/XSJnHcL5j/UudLPxdO40VHoQwaA5EaKjwsvKSGkf5YDyhDYFGQ+6Ke0aZm7cORclqYAioyevuzfvZmp5gAN48GKXEI2io8ILwsDUtYZ2NJQVukfusPRHmq5ceyd0z0g1Njsc9dSSjYb+2pbMfkJw/uQjDMGkJRmms9NJQ7kZKmdPzIf/8DaQY3EkGmD2liJsunIrHgNXPd7A7HKPJ78NjwJoXMoV/bY7RV4YhGLUIRge4a3MHqy45feIvx4o2BBoNuStujwH3vKACuA2VPmcyiqUkbaGoM/+4DbUiDcUkbx2K8tg/1LP4j3sJx5R32vZO9MRSrPqPvSy/aBoTvEpU8Z7NHTQHIo4OEnLoiXW0ZAsdLbsne4yFgrzZbpjmQCQt/aBiIkIINTFbqgYAyOn5sO6lAzljyc/oKvUY/OBzM/j+K4eY/2wrlgSXIWj0FyGl5LrfttAbkypuIKGhwosQcdXXWkLogJKeaC8QnxkPaEOg0aTJXnEvv2g6dz3fQWsgwl3P72PlJTV4TZWb3hyIMqPMzTUfnszC53aTQqmU3vTvGSMAmQWrBLYdjvK1/9vKh6b4uPmiWscNZU83ScsatOqF0ZUtNJR7qtAYPQbUl3uclX32zmfZRdNp8vvo7RyAdPGXaQh6YwP0xi3eOjhAQoLIkvZuD8WOMDLojVv0xi22HujPcRdZUhlbu3LYbUBjpYflF9c4bjgppaoQD8Zo8o+v+gGb8ecM02iOgbgFrcEIwajF653KZQCw7KLprP+f9ZiGwYpN+5jgEZR71OozGBmcp5I/pbQE1YTWVOml0mc4r/fELO58vgPLsnI09bNX1kfreZD9vlOF3YpzqBTPlnTDnjUv7Kc9FKeh0sfiC6flvB63YMXFNfzzF5p4/6RidofjpFIWoKQfElJp+NgZVx4Dass9OeMYesEunP9X+gwaKn05r5Z4TZZ8YrpT+KeSAwxWXVLDxn9oPK3Sco8HvSPQaArgMcDKamr+5oF+gpEEG/96kOZAhFDUciII5T6DYrfANFTXs55oCiv9msvIDWjWlLqwLItbLpzClb9uc1oj2q0xb/+P3XjdLqe6ddlF048pW+hkyTREk9YRC9sKuamyXUaNlV7VyjGgUmIRyng0VHiwLIuGtN/fnoB398QdX322018Y0B6Oc9fmDgygPZhbCHbu5GIWfmQii/64h764pKLIxeQSD7OnKjXZVLpFKVaK2VOLeePAACBozHL1ZTPS8ZeRRhsCjaYAqo1hJgcxacGiP7TTn84mspFAKFq4xVl2VsvZfhcdvSne7krwtV+0UOoxHN2ibN48FMNtxJ24QdzCkdMu+B3piTk728ae0PMbpRwti2fpczvY0dkzZGHbUMZGCJHu75Di0f/qZPEf2klZUq3IKzy4hWRXd5SemEVzd9TZsdhuIyklM8rcWFLy9qFY2sCqmEEw0o8pVBaW/dco9xqYSG750z5m+n3c8PGpzKyZQiAQYMXFNQQjCb6dLibbcjDGv3yxCSEE9/9lP23BqNNycjyu/IdCGwKNpgBeUzCzqogt+1UFqxBqwndnGQeAMq9Bb1q+4Eg0B5OOn1qifNpDkUwrYyYtiQuLaFIMyi4SQuRMzI0VHhoqPPREk/TGVA+GJRdOc9wflmXlyDLkT4SxlGTnoT5V2JYeo+3myQ7yFsoaklKy5oX96eB3yjF+Ani3O8qd/9nhpH6GYxahaJKHX+6kNRihye+jocLHmwcHkFIywWtgCpVpZXeGk1L1hGis8HD9x6fhMWDBc23pTJ8Ij/71IA/VTCmo7SRRxrTIndl9FCrYG+9oQ6DRFEAI4fQxkFKy/uVOWoMxGiu9pFIWbaEYjZU+ll80jbtf2M+bBwZyVq2g3EIpK/O4zCPoiatJvsxrgLQVT1UQ830Ti9kTjhKI2pNmiit/1YJpGPTGLSfLxp6c83Pu131mBov/0E4gkmJr5wDzn211urHdtbnDkWVoDkQHTYQeA2ZWFRMaSCgDJKCh3D0oyNtY6VUr+UovHgPn/NjBb7chqPCqTCqVMWXxdpbuf6nH4KEX9/HGYfWLv34gSpkns3OyjarfZ1Dqhp6EkogIR1KICtURbM0L++mL27sKaAtEueXZHbx7SMl/L/3kNCp8qi9Buc90egqMhoK80Yo2BBrNENh9DIAcMTIgx8Vy26W1zoTo9/u55ddv8ebBCMm0oFmJR+0uln5iKnf+ZwdtwQiWZWHm7S5u+eRUeuMWC55tc3YP4bgEUriECn42Vnq57y8d7A7FmFHhpTFrciv3mk5WU3Y3tnAsRVsw6uxsGioyE6HtRnrgLx1sTRszgFKfyYKPTeWm/7fHcTeFYymslPo9rVTKyctvrPA442is9HLdhydx4+93O66vFGrsKQuQlmMEbKTMBH9LPAamYVBf6aU1EHEshIXaofTELVqCUacnRKnHoKHSy67Dfc44EzLTRazMYzgFYiNdkDea0YZAozkG8oOJ2T/bmTSxlMQwDPb2xJ1A8nmTi7n5k9Oc1/f2xulJ2O9UK+9Sr5rgH3ixk7ZgJKdJygRTuTYQ0FhZRDJpsfWgKlrrjgzwgck+Hv27+gK++v20hRM0VnjY8MoBemPKmJw7qZglF05V3552Ldkunezv7Yul2PBKp7MDsCQs+n2bEyR/I11IF4qpN234XAMAD/ylg5v+3x6aKr28eTBCCpUB9L1P1XL7n/cSzLSDdjANwZP/qxGfy8BjQG9CUuoW3L15P3/r7HeOqy/3UOoWqs5AQqPfy+ILpiKlZOHv9yhFWKl2N9ldxIb6G2oyaEOg0Zwg2b76sycfprHSS09UKZcKIR0/vddUq/FAJP0acP7UCSz++GTufbGTNwuoncasjFxFW0gFWrPjEW8fjqodQlgVR4GgORhlZqXPmZwXPNeWlluGbYcHuOKXLZw70cctF9XkuHRMIdWuAbUQbwsn2PC5BmIpyeLfqyY9NikJ/Qn1mQ0VHuc8vHFQ6fv3xiKcN6WI9lAMieB7/7kPiaDCK7Ak9MUlFip//YzqYip8aiqyDVNDpY9FH5/EFb9sI4XaWd38yems/UsnbUElDLjsk9NY+5dOdnUPOAaqP24RS0mKdGL8caENgUZzgmQHUd893M/9n651fPVtobjjjxdCsPKSGu7a3EFr9wA15T6WXjiZq367O6cQLRt7lS6A+nIvScsilYzSm8w4ld46qHYR4UjEUdx8rXMAy7Io9rhoSq/qswO5Ww5Guf/F/Sprx5IkUxYDSTAMmOAxHfE7n8vA54ImvxcZUDsDgaQvrprC+4tMLEsZm7pSF5koieCGj6udx5I/7qY7kqLSZ1DvL2JPKMYHpvlY/PHJJKSg3GsipeRQf4ItnX0kLAhE+rnut20YhsBA8v7JRQghMoV4Ik5vQp33YE7W1qmroTidOSFD0NfXx4MPPsjhw4ednsUlJSWDjvvyl79MXV0dANXV1dx6660AHDp0iIceeoje3l4aGxu5/vrrcbm0bdKMLbLz6M+aVOL46lsKdLoyDIMVF03j279q4e1DEa78TXtOBpEr3Ws3lWUATAHnVHtpCUQJp481UX7zIrfBQJYvPps9wQgz/MUsu2h6Wr2zjUTWd7UHo2z4+0bue7GTrZ39tkYehoC1n57BxGKXE5RedMFUYqmMy+atQwO4DUFduZeWwADhOIQGkhjptp1FbsGi36k6iYGECpD3xizeTu8YLBlh/X8dpD2kAvDvHo44v5tNb1INyCWgNRBh3YsdOTGRMo/hGDmJQEhJk9877iWl3wtCnkAZ4k9/+lNKSkqYN28ev/71r+nr6+PrX//6oOO+8Y1v8Mwzzwx6fv369Xz0ox/lwgsv5PHHH6e+vp5Pf/rTx/Td+/fvf6/DPmaqq6vp6uo65d9zKtBjH17sHP3pkyfS3d19xJz9wEDcyXMHFQfoz5rFc0PICpfIbbBuU+kzmFHh440DA0OuhT80tQjLgi153bk+OKWYGy6YypI/ZNw+LgHFLoEQksZKH6surWXtXzodgbj3Typidzie4ybKZ6ix2tiN5e0m8pVFZsGqbMgYPLsh/T9/YSaGYTjn1T7P0yZVv+de0SPJcF/r06ZNK/j8CZnOV199lYsvvhiAiy++mFdfffWY3yulZNu2bXzsYx8D4JJLLjmu92s0owk7YAzkyCsfy6TkMg0na0aggsf55E+s5V4Df5HJGVVF3HbJdGZP9lHmHixpAbCrK8Lbh3ONgAk0ByIs/n07KQlVRSYfn1HBw5+toychCceV+2jVpt3sPNynpB8saAupvr8Fz8EQYy10ZDK9/fAXmTTmfV72gr7Enf1YDJK4sB9nS0Zojp8T8sOEw2EqKysBqKioIBwOFzwukUiwdOlSTNPk8ssv5yMf+Qi9vb0UFxdjmio9z+/3EwgEhvyuTZs2sWnTJgDWrl1LdXX1iQz9mHC5XMPyPacCPfbhR0rJ8t+9w46DvZw1qYS1nzun4MTktywqivYQiqj8/3DcwmMKXMoBT5O/iLcO9A85oX64tpx1l88ibiljI6Vkd2972pVSACHIDjELlBupLyGxpPLdr593Lo+9vJuVf+7Ieeu2rkyqp9uAsyaXce/nzubW597hb3tDJKy0lIap2qvZMQi3AcUek95YCivv9xACqie4OWtSCXd85ix8LoNbn93OzkN9nDlxArf/jzP5yj+/TjiawnC5+EhNKTsP9XHWpBJqpkwqeE7H6jUzWsZ9VEOwevVqQqHQoOe/8pWv5Dy209cKsXHjRvx+PwcPHuTOO++krq6O4uLi4xro3LlzmTt3rvN4OLZTY9FFYaPHPvxEkxY7DvZyuC+OZfXQcfDwkP7qp+c1Eo6l2PDXA7QGY9RXeGnpHiAUk2k3jEIAT/zDDB555SBvHFK5l61d/XQe7sbnMugHQtEkwUjCad9Y7oGaMjfbulSeak9c5oi0yfQxEkFfXGnxX/GzN4/4u5W5ob7Cx7uHern512+x7KLpxFITHT3/B17sZOuB7Kwnwca/m8H9Lx1wnncJKPOZ1Fd4ufkT0yhymwz0hBgAbrlgMrHUJLymIBwOYwqBRLksrv9wNUJMxGsKuru7C45vrF4zo8U1dFRDsGrVqiFfKy8vJxgMUllZSTAYpKysrOBxfr8fgMmTJzNr1iza29v56Ec/ysDAAKlUCtM0CQQCznEazVjEawrOmlSCZfUctXrVMAwqiwynUM1jwF3P7+P1zlxfv8uAEp+HOy6rG1IqudQtKPOYhGMpXAaYponb5QacgoWc3YXbEDRUFrG7J0ah1u4GmV2DTU9CSWmnyFQm28V20aRFezhGbqhX1VTcfmkNdz2/L12J7cWylKT0gy93svziGufo7Bx/r6mylAhmMpe0y+fUckKuoTlz5rB582bmzZvH5s2b+fCHPzzomL6+PrxeL263m56eHnbu3Mnll1+OEIL3ve99vPLKK1x44YU8//zzzJkz50SGo9GMKEII1n7uHDoOHj5i0DI/kGxPgHZq6ZsHBkhYEhM4f0qxMxGuuqRwL4A1L+ynL5ZyJvBAJAXk6vdXegVSGBgCGip8vDGE68kATAN8JsgU9OVJIrkEjpbRsk9OozchKXExyP0jJax/aT8rLqll1aW1jjDegufajqr3Mxraco43TsgQzJs3jwcffJA///nPTvooQEtLC3/605+49tpr6ejo4PHHH8cwDCzLYt68edTUqJXA1772NR566CH+7d/+jYaGBj71qU+d+G+k0Ywg2UHjQhxJLtrWxbflKgpp/+dj1zDYq3cT8BeZNPl9NEpJSzDGzEofSz4xzemJbFkW3/y/zZCWdnjfpCK2pfWALJnR/Sl2m5BV33DORB8dvXGCUYvm7gjf/lUrPfGUknHIG2JSQqtTQ2HgNSGalNSXKRnuhnL3EXdMugp4eDkhQ1BaWsptt9026PmmpiaampoAOOuss1i3bl3B90+ePJk1a9acyBA0mjHF0fr+ZusbQW6PAFvbJ9uA2DUMvTGlt3/+lCJu+sT0nIbw2atqnwFSCs6fXOTUOSy+cBrzn2115CJAGYNwLOWkspa64XufqmHti520BKLUlnl4My110ROzOG9yEX1x1ee52G3gMgzHPeYYv0CEcCxF0oLmYNwxdvmN7PUOYPjR1VsazTBypL6/kOs2ApzdQ325h/ZQXDV7gZxq5RWX1A45kQ7VA1mmM8clsO6lA/QnBvuJhIAPTCnmnz482Skusxu+CGlR5jUceWm3KfjnLzQ5xim7sU00qYTismsPwrEUPXGLcq/g7uf3sfVAPyCYPbWYFbpXwLCjDYFGM4wcyf+d7zbKbvEopaSh0odEOkJw9oo6W/TuaNj9eW1J6q2dkUzBlikodQssBAL1fQawfNNeGis8WAi2pAvXth6Kcf7kInaHooRikrZgnLil6huEEGTrvTnGL61gmrSUnHSZx3B2SModJWkpIJGtOfVoQ6DRDDND+b/z3UZATgtIme5a9m5XxOk1YGfeHGubylhK0h6MOeJykFYGEjCntoLvfqTaWdHbwV3bENnH2uzridPkL6I9FCclJYt+31aw6U228XMLSW9COvLQXiFzXFtN/sKtJDWnFm0INJpRQr7byOcynAnUybiJphCqdiunmf2R4g6DvsPvhQDUV3oRSFqDMRoqvNz39+c4RZ0qliAzhqjCA0LQ2zlA0pKYQhmn5RfX0BO3WPT7I2cDZRu/CjP3+SO5tjTDgzYEGs0oYSi3kd0OUrlXcHoKZ8cYcnYOWW6jo30HZALKdgOX/GPtSVplG7WkdxCCxRdOwzAMyr1iSJG9Y/298/sra4YXbQg0mlHEUG6j7AncY5ATjIVMg/t1L+5nwXNtR3QRHanJTiHsdpX1Fd6sZzOGRuf9j320IdBoxgjZE3he8y0naNwaih2Ti+hYyYlbSDhvchHt4bjjuio0Ns3YQxsCjeY04WipqSfjM5ddNH3QbiSbI8lva0Yv2hBoNKOQ9zKhngoXTaHPzN+NZI/5WLOXNKMLbQg0mlHGiUyop8JFc6yfebSqac3oRRsCjWaUMVYn1FPhmtIMD9oQaDSjjLE6oersobGLNgQazShjLE+oOntobKINgUYzCtETqmY4OaHm9RqNRqMZ+2hDoNFoNOMcbQg0Go1mnKMNgUaj0YxztCHQaDSacY42BBqNRjPOEdJuPaTRaDSacYneERyBpUuXjvQQ3jN67CODHvvIMFbHPlrGrQ2BRqPRjHO0IdBoNJpxjjYER2Du3LkjPYT3jB77yKDHPjKM1bGPlnHrYLFGo9GMc/SOQKPRaMY52hBoNBrNOEfLUGfxX//1X/yf//N/6Ojo4J577qGpqangcVu3buXpp5/Gsiwuu+wy5s2bN7wDLUBfXx8PPvgghw8fZuLEiSxatIiSkpJBx335y1+mrq4OgOrqam699dbhHqrD0c5jIpHg0UcfpbW1ldLSUm688UYmTZo0MoPN42hjf/7553nmmWfw+/0AfOYzn+Gyyy4bgZHmsnHjRl5//XXKy8tZt27doNellDz99NNs2bIFr9fL/PnzaWxsHIGRDuZoY9+2bRv33Xefc4189KMf5Ytf/OJwD3MQXV1dbNiwgVAohBCCuXPn8tnPfjbnmBE/71LjsHfvXtnR0SFvv/122dzcXPCYVColFy5cKA8cOCATiYS86aab5N69e4d5pIN55pln5K9+9SsppZS/+tWv5DPPPFPwuK9//evDOKqhOZbz+Mc//lE+9thjUkopX3zxRbl+/fqRGOogjmXs//mf/ymffPLJERrh0Gzbtk22tLTIxYsXF3z9b3/7m7z77rulZVly586dctmyZcM8wqE52tjffvttuWbNmmEe1dEJBAKypaVFSinlwMCAvOGGGwZdLyN93rVrKIuamhqmTZt2xGOam5uZMmUKkydPxuVyccEFF/Dqq68O0wiH5tVXX+Xiiy8G4OKLLx4VYzoSx3IeX3vtNS655BIAPvaxj/H2228jR0Fuw2i9Bo6FWbNmFdwp2rz22mtcdNFFCCE488wz6e/vJxgMDuMIh+ZoYx+tVFZWOqv7oqIipk+fTiAQyDlmpM+7dg0dJ4FAgKqqKudxVVUVu3btGsERKcLhMJWVlQBUVFQQDocLHpdIJFi6dCmmaXL55ZfzkY98ZDiH6XAs5zH7GNM0KS4upre3l7KysmEdaz7Heg389a9/ZceOHUydOpUrrriC6urq4RzmeyIQCOSMs6qqikAg4Fxbo513332Xm2++mcrKSr7xjW9QW1s70kPK4dChQ7S1tTFz5syc50f6vI87Q7B69WpCodCg57/yla/w4Q9/ePgHdBwcaezZCDF0n9uNGzfi9/s5ePAgd955J3V1dUyZMuVUDHdc86EPfYgLL7wQt9vNn/70JzZs2MDtt98+0sM6rWloaGDjxo34fD5ef/117r//fh5++OGRHpZDNBpl3bp1fOtb36K4uHikh5PDuDMEq1atOqH3+/1+uru7ncfd3d1OQPBUc6Sxl5eXEwwGqaysJBgMDrlqtsc6efJkZs2aRXt7+4gYgmM5j/YxVVVVpFIpBgYGKC0tHe6hDuJYxp49zssuu4yf/vSnwza+E8Hv99PV1eU8Hs7r+0TJnlw/+MEP8tRTT9HT0zPiO0iAZDLJunXr+OQnP8lHP/rRQa+P9HnXMYLjpKmpic7OTg4dOkQymeTll19mzpw5Iz0s5syZw+bNmwHYvHlzwd1NX18fiUQCgJ6eHnbu3ElNTc2wjtPmWM7jhz70IZ5//nkAXnnlFd73vvcNudMZTo5l7Nn+3ddee23EzvPxMmfOHF544QWklLz77rsUFxePGbdQKBRyYkjNzc1YljUqFg5SSn74wx8yffp0Pve5zxU8ZqTPu64szuK///u/+dGPfkRPTw8TJkygvr6eFStWEAgEeOyxx1i2bBkAr7/+Oj/5yU+wLItLL72Uz3/+8yM8cujt7eXBBx+kq6srJ320paWFP/3pT1x77bXs3LmTxx9/HMMwsCyLv/u7v+NTn/rUiI250Hn8+c9/TlNTE3PmzCEej/Poo4/S1tZGSUkJN954I5MnTx6x8WZztLH/67/+K6+99hqmaVJSUsJVV13F9OnTR3rYPPTQQ2zfvp3e3l7Ky8v50pe+RDKZBODTn/40Ukqeeuop3njjDTweD/Pnzx8yjXq4OdrY//jHP/Lv//7vmKaJx+Phm9/8JmedddYIjxreeecdbrvtNurq6pyFzD/+4z86O4DRcN61IdBoNJpxjnYNaTQazThHGwKNRqMZ52hDoNFoNOMcbQg0Go1mnKMNgUaj0YxztCHQaDSacY42BBqNRjPO+f8BL3PB0bq2pggAAAAASUVORK5CYII=\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 生成月牙型非凸集\n",
    "from sklearn import datasets\n",
    "x, y = datasets.make_moons(n_samples=2000, shuffle=True,\n",
    "                  noise=0.05, random_state=None)\n",
    "for _,c in enumerate(np.unique(y)):\n",
    "    plt.scatter(x[y==c,0],x[y==c,1],s=7)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAB9KUlEQVR4nO39d5gc1Znoj39OVadJ3TM9o4DyjIRt8JogS+RkkHH2srvevfbu4rQ2Bgl2iQaBBGtLAmHSrgnGa2PWa+7vd9d3rxM4gYzJYASSyBgkjSQUJ/R0T+xQVef7x+mqrk4zPZqgCfV5Hh40XelUVff7nvNGIaWUeHh4eHhMW7QjPQAPDw8PjyOLpwg8PDw8pjmeIvDw8PCY5niKwMPDw2Oa4ykCDw8Pj2mOpwg8PDw8pjm+Iz2Aw2X//v1ltzU1NdHR0TGOoxk/puq9TdX7gql7b959TT7mzJlT8nNvReDh4eExzfEUgYeHh8c0x1MEHh4eHtMcTxF4eHh4THM8ReDh4eExzRmVqKH77ruPLVu2EIlEuOOOOwDo7e3lrrvuor29nRkzZnDFFVdQW1tbdOwTTzzBz372MwD++q//mnPOOWc0huTh4eHhUSGjsiI455xzuP766/M++8UvfsGHPvQhvvvd7/KhD32IX/ziF0XH9fb28j//8z/cfPPN3HzzzfzP//wPvb29ozEkD49JjZQSmUriFQf2GA9GRREce+yxRbP9zZs3c/bZZwNw9tlns3nz5qLjtm3bxnHHHUdtbS21tbUcd9xxbNu2bTSG5OExaZFSYt27AWvNJVj3bvCUgceYM2YJZYlEgoaGBgDq6+tJJBJF+8RiMRobG52/o9EosVis5Pk2bdrEpk2bANi4cSNNTU1lr+3z+QbdPpmZqvc2Ve8Lhn9vMjlAx95WrHgnmq7RVFeLCFWN4QgPj6n6zqbqfQ3GuGQWCyEQQozoHCtWrGDFihXO34Nl/k3lzMCpem/l7ktKCekUBIIj/g6NxfkqYbjvTEqJNa8ZTAtrXjMdPb2I3r4xHOHhMd2+i1OBcpnFY6YIIpEIXV1dNDQ00NXVRTgcLtonGo3y5ptvOn/HYjGOPfbYsRqSxyTDNpGwazvMb0ZcthZNO3xrpnO+3Ttg4WK0VTeMmzIYbEyFikkIgbbqhnFXWB7TlzELH122bBlPPvkkAE8++STLly8v2ueEE07glVdeobe3l97eXl555RVOOOGEsRqSx2QjnVJKIBGDN7Yg71lX0l5eqWNVppKwezvEO2H3DnXMEXTIDuYLEEIggqFhKQHPwexxuIzKiuDf/u3fePPNN+np6eHiiy/m7/7u77jgggu46667ePzxx53wUYAdO3bw2GOPcfHFF1NbW8vf/M3fsHr1agA+97nPlQwx9ZieSH8A5i2A7i6QEt7bpZRDMJTbp8JZvpQS+YPbobcHfH5Y0IL8we3IPTthQQvi61c7gncszEf2OaU/gMikIRBU19i9QykmKLq34Z5/oq12PCYPo6IILr/88pKf33jjjUWfLV68mMWLFzt/n3vuuZx77rmjMQyPCcZIBKqUEnnfzfDebqgNg+6DhYuVAHVTqTBNp2DPTjAyEInCFy+F9Veq43q7kWsvQS5cglh5vbruKApUy7Kw7l4H77WCZSKFBouWwCWrYd4ipeRK3dtwGEWl4jH9mLRlqD3Gn+EI9hHPUG3B1t0FkShi7V2IukjxOQJBJURhcGFasJ+oiyAXLlZCuK8H4jFgB/R2j6pAlVIS3/BNeP1ldS0h1P9374B71sPeXbCgGbHy+pEpnEqfg4dHCTxF4FERlmUh7VntoiVDC/aRzlBLCG7I2vkPw7Faaj+56gZlU//BHbBHKSxqwyMSqEXKMp3C2PVOTgnUhkHTYf4i2NOq/B+gzEUjUDieg9ljJHiKwGNIpJTIe9bDG1uUQNu1fWjBPsgMtZKVRaFgA/JWGGLl9Y6tXQhRciyF1xFCILO2eWl/FqpCXlpaQQz2PNzjsm3/pFPIH96RtwoiEMTX8n4yibjad9HRiIuugUAwZ4JasFidU8oRCfByz8HDYyg8ReAxNOmUWgnYs9r5zUPOlMvNUIdjMnILNhXxk11hZBWTfK+17DlKXQcoee1SAtQW6NK1nxOV8x+3KX/DgsUgUOeTFiCUmck0nOcmgiHqr/wW7Sv/Tpmf3mtV19Q014rkduTalXnXGowjkQvhMbXxFME047CEiHt2P38R4tK1FR1bagY+mMlosLGpCKJFShnNb1aKqeAceTP1guvInoT6vAJzlUwlYde7kOgCy8Tqjiufwn03Z8NZuwAJb/VAdQ10x/NPkI1KspWlCFXBwiXAjrzVkRAChFCRSxU+jzwFt2Ax4utXDTvM1MOjEE8RTCMO14F7uPbnkrPyMiajwcbmRBBlHausWgPfuyXvHFJKrHvWK0G9oBkuXZu7jmUiv325+ntBCyBhfotjjgGce5NSIu/fmBX2KCF/7T8hjzlOha8mXCVQTEOFo+o+qKqBvm51vpo6FY5axo8BOV/HsJ+HW8H1diNvuBjZfLQXLuoxIjxFMJ0YgQPXNp/Y5pGKFEKJ64lgqLRScZt+LAsrOYBeVV3yPJqRQa68XkX41IYRQmAlB+CNrUo4v94F965XCiPWARu/qQT4HgFHzQPTgl3vItdcoiKHQJl65i0EiTqPG9OAt14tvj8pQZpQVw9r7oSH7lXKYuFihGtWL5MDav9AUJmCCvwIJZ9HuXcVCCqTVG+3CoXt7qrMZ+PhMQieIphOlJh9jmlIaJnZbkmnZjCkTD7xmBJuN1yMdft/omlavllo4WKkP5BztM5fhHXp2uJr794J925QJiSkyh0wDHjrlfz93uyG6jp1zXgnyujvRoCuq3/atn9nkwaaBr0JuOEbcMzx8K17IJN2snutezeoAnJzFimlsacV+nuVEM8qVVFKgLufXYEzWXz9KuSaS5RyG8JnM9T79fwNHuApgmmFbaKwI2Ick0ulgr3MLLVUFI0tWCo1KQkh4EuXwTVfVrPynjiyow2raWY2saxVmXwuWZ0f65+IqbDWS9fAB46Dt19VwnFBS86PUN8Is+fA268VX9gwlBJwcJVnqAkrvTC/WQn9t7blTEmglEB1DfQklGDftV0phN5uZF0EueH7sGs7ViIGceVvQEoVPurzQ18P8j9uR2qipANbrLxe+TZ+fHe+MzkYQi5cospl2KYytx/GvpMhFLeXjexh4ymCaYhjmpi3SNndKzUVlVlR5JyXLUoIFwqWSk0WtWGoqVOCHuCmVcijj4F331KC9q0euPvbyPd25WbnUsKb25QpaN9ueP+H4BvfVGGh992s9pmzQI0J1Pje/6GsUnAJdaFBVTWkBsA01Wd92XG8/Sp8cCl88EQ1o7cVh2moKCGb2XPg3WwRxZ4EJAdUNJEQufEKAe//IOzfmzVX7VDKJq58DzKVxJJSmcoeuk8J+96eEisIe+wCvncLVilhPpQp0MtG9sjiKYLphvvHL6WaUQLMX6Tq4AxyaMmkrIKwTrdQK1UXqFQUjEwlMfv74L6bs7Nln5qpG5msbd41qrdeVTPrQuzkrEQMfnAb4p9vgpXXq9XCW6/kK45d20vcnVQmGzc1YejvUcfs2Qnr7lOC/IZvKEEPYFm5/VvfcT8s9Z+mF6widPAFlCLdI5RfQtPVPS5YrEJTX3tZjcfOQvb51T693cj/uB0uulqNJxGD3dn9EjEgqyjs3gVDZBsXmty8bOTpi9e8fgpTshqlLRzqG1W9m1VrlDDY04q87+aylSsdJzHkhyu6z7dwMSxYov6dtWvb5ytVadOyLKx//xbyX/6ejgs/Bq++pGbbhlF4dQg3wAc+VDwwIeB9f6GUhroQvPkKVnJAJZy911ps20/2k7ca8PnyhTUowfsvN0EwK1S7u2DNJWqmfuuPSo/FMJTQDjfAh5YhwvUqg9iNacB7O+GfrlTP/b1d6jbW3Qdfu1KtAOyxSQmRBmXyslcVb21DWpZ61pGoWnH09ajtvT3I+27BNIzcu1p5vSrPUVDCojASa8QlLjwmNd6KYIpSzv5bFMqYTmHt3ZWdXe4oaR4YzJZc5HfwB5Rp58d3q0JuC5ag2Zm7rpWDleiC//yuylYeCt0H125UZqO1l6jZuO6DuogSpjv/nD+bt0z43s1Y//ItmLswP+SzkOpaJVQLnciWCTdflf9ZT1z5MGrD8K171b/dqxOfH449Ab54KaIugqZpmF+/Bq74h5y5CaA7AfffAvv2qPDUPVkB/MCd0F/QgGZeM3z9arjqi+pvIwM/uB1x2Vq1Qlh3RU4JGhnlx1j1t1h1EXVfLlOdHMRsZJe48JzH0xNPEUxVBrH/5mXsVlKsbAgnseXzg51tKy1lb7eTrnq35mLm5y9SSiLRBdf+U2kTD6goHdNUwr42rM65dmW+rd00IJNW/xWadECZkO5ZD5ethau/lG/Ld5McKFYCQ9HbXawEqutgUfY5rr8CuWAJ1io1yxbHnoj12ku5fS0zF46q+1Q4KKhnbGTUKsCylMLbt1tlIR9zHLyeLfHxXqsS3E7hPAsScZyVhGVmFTsuU93QZiPPeTx98RTBVKXCapQVRfYMJTRMI2czt+3aLqRlwd3rlG3eNAFZXgkAtLwPDuxTAvIr/wLrLi+9f18PvPN6+fPs3q7Gdd131EqiFIONYzAKj+vvUQ5oTVPCvHcr8u5vw3ut6C3vx9L00teSUpmEAkG1XQiwJCw6Wpm17Hf3taudiCQsE8vnR8++Oys5oBLsbIWm6fnF83p7lNnoB7cjL12TF5XkzsXI8/eA5zyeRoypIti/fz933XWX83dbWxt/93d/x6c+9SnnszfeeIPvfOc7zJw5E4CTTz6Zz33uc2M5rGnBsEM3BylrMKST2E1NnTKRmFmhtnAx/PD2XMG6Quw+Azv/nPvs3bfU/9/cCj+6U5l3Css42AzWjasnAd/8ilqhVEpNLfSVWGFUgmUqRWDzujJ7GVtfKD+G7PhFJq36FNj1nP5xpVIoNXW50hYD/eqY3m64dz3ysmy/jwfuhP3vqcgmKWH/HlUKxDbZrblE+Tj27MxbzblDh+UgWd8eU58xVQRz5szhtttuA1QZ42984xucdNJJRfsdc8wxXHfddWM5lGlJpVU53Z8P5gvIO1cgqMJFe7vVikD3QahKCbx5i+DSNUq4pZKqAYxbYPv86hgp1f/dSqCQt15V59Z9xU7fSpHW0PvYHK4SADUTN031/yXHwtsuk5MQQHa1lHcvEn54hxLECxarBLfuLrj2q6WvYa+49rTmKqS+sTW7ClGze8cnkk5l/Q69efWPpJQqP6HSrG+PKc+4RQ299tprzJ49mxkzZozXJacdlfSsHaxPbp4vwHYclzm3ynC9GmrrnPo6aLpyqO7dpZTAD++Ab1+ev2oIN9gnzArHCjCNw1cC44ll5pTOly5Tz8QmVKWejzusFLLhrO8iE13qHocyVX3guFwy2v0bkQP9+UrWXolZpjLJ7d6ulEStqn8EKttZfvtydS072sud9W0XCvR6H08bxs1H8Oyzz3L66aeX3PbOO+9wzTXX0NDQwIUXXsj8+fOL9tm0aRObNm0CYOPGjTQ1NZW9ls/nG3T7ZKbcvUkpid9yHcbOt/G1vJ/61beWLimQHFAlD+KdaLpGU12t40CUUhI/+hiMnX9W55gzV9XxsSziG76Jsesd/Is/4Jxb7X8sxo630VveT+bdN1GzXotoQwNde1ux8rJ2Ifrt79L70P0Yre+oyKGiUNERUBPOJYEdSSwTVn8t/zO3Q9vIZHMDRDazOa5MWOVw+V00XceqC0NXp1oJ3HCRUsaahr/5aDI7c01w/D/+dzK9PeDz4Vv8ARoaGwHo3NuKlYghok003fGfiEhDXl5HJd8jmLq/s6l6X4MxLorAMAxefvll/v7v/75oW3NzM/fddx+hUIgtW7Zw22238d3vfrdovxUrVrBixQrn746OjrLXa2pqGnT7ZKbcvclUEuvdNyHeSdq06Ni/r2QNGykl1rxmMC2sec109PQienMhi/Lr16iuWoEgnZ2dav+71zmtFtPGG7Tv24sWqlLbUil1ruSAcnIioTtO58bVkErlX9znI/avlysTxZUblBArRTnH6lD09xz+seNJXb3K4bhwFaz+evnVTm0E1t4FP/6uyp4GrDe25q80MhlAwA13kvEHlD8AINFFZttm51kY216k/ZK/U+UyZqvCe3J+C50ZE9GZW7FV+j2Cqfs7m6r3BcpcX4pxMQ1t3bqV5uZm6uvri7ZVV1cTCqkv2tKlSzFNk+7uCTCrm2wUJnYNESWkrf9eydozJatg2k1pAPp7kT+4I7fv7h3KJv3eLhUeas9e32vNOTd1H9x0txJs8U5VVqG2TuUBlCJUrWzdQqhjbXPSUEipwi5ts8tERZAzEVmD+C8sE/7399Szrc4KfylVyGtNOHeyeYvgx3fDjSthwJWHkGcyMtR7ev1lFd00d2HpJLIKv0ceU4txWREMZhaKx+NEIqop+fbt27Esi7q6upL7epSnkight51fCJFXTsJxFO/aDvObEZetRdO0nGCQUoVrGhklyO32jHYtHcuEi74JD9yRUwpvvmIPDhpn5JzLvT3Kifmte+HKfyy+GV2D7zyo9n3w31W0S8XI4TmHxwO7zIT9rLrjaoa/7goVZWSWGW8qCa3vKr+LGyOjit0B+HSljHuz4bsIZXaSUtVO6i0xqTINx49TGEwwWO8Ez3k8dRlzRZBMJnn11Ve56KKcGeDRRx8F4Pzzz+eFF17g0UcfRdd1AoEAl19+ufeFO0xKdgTL4jRueWMrmCbS50MeeyLapa6GJ7u2q1ljdxfy7m9jfeNaJ5KkqMl7IKgElR3yONAP/3qp6pq17j6kzw+X/Z198VxIqZ0Fu3sH/OA7pW9koF+N5z+/q0JIh8tEcnJqOv7jPkxm1w6YeZRa4bzzeq6XwFDMWwhvJ4rvyQ6nNQyXEkAplm/do97P+itzn4cb1Ln+/HourHeQVaMdYuolmE0PxlwRhEIhfvSjH+V9dv755zv//vjHP87HP/7xsR7GhGc0UvsH/eHaZhzbHm1kYPe7uWzTQFDZj7u7lNB5+zVVImLhEmVKKmjyLqVEfv87asZpz0DjMSDbA7i/F2k7gk0DvnsT7N+XG2xvN7ztSgarqctl/xoZlRVsV/KczBx9DKZdDrsw52IojjkO/ukq+P5G2LsHyJqFbGVajtuuh/ktLl+JgIVLEJdm8wrSKURdpDKzoJdgNi3wMosnAKMx8yoXG+78cG0TzxvZuH8E9GXt/dlVgbhsLfKedaqSZ19PVrDn6g/ZKw4rOaCE0xvb1LnDDbn6//NUFVPp8+fHy7/9ulIYNvbnuk/ZzNPJ/Bt6943im6wNlzZ1TGRMC23OAqzO9uEdV12rnO1X/EP+5/oQP1nLzL5/mUtiq62DC1eqfz9wJ+zegXTVHir7/fMSzKYNniKYCIxw5pVn35dWSUefEALt0jXKxJMcUGYDux5+9nqapiEvu7G0GQiXeenNbfmz0nkLYeX1qiPY3l0qY/Wfriw2Z5SaydrNWmDopLHJpgQAtr9J0R3ZeQSD3Wt/b2llWHiMphU7nCMNanVnAW+8rJ7btV9FfnCpUtbuhLNgqOz3byi/07DalnpMaDxFMBEY6czLHb0TicI3NyIaZxT9OPNKSSxcUiTo7X1EgRmoyFRgC3QhlEln3x6VCdvbnRPqyYHKwjgLI1tGk+YPQOvbo3vO0UBoKjKqXM6DXXRvyPOUiTqas0Ct6uYtyH0mpSp/Pb+lyEdwOH0J7HwD6903Pf/BFMBTBBOA4dQFKolbkVgm3HotctGS/LLDFJigFrQg1t2X31ugYExFqxK3eQlgyTFwYG9+mWchlCD68d3Du4fhUomwnIhKANQsfrB3XHhfQqgIqxtXkddHobo2v6pqXUSVrd63O+eIts1pQvkJ+NpVStnbpj53X4L5i9RKDleE2Q9uV1FbJXxOxs63Pf/BFMFTBBOEYbV0LHGstuoGZE9ClQ4o11ug0ASQ/cyOMBrKYS2EUIXM7l6n2jduf0uFKNY3qlWC0JQi2rurfNlnXYdQjZrJDoygrk8lM+aJxPs/lHsuwSqYOx/+nBjyMEA5jWcepVplvrk1V55j/iJVoM+u9TS/Wa3O+rPPXlpw23+qa/oDyj9w4yrkwsXqPdp+JTtarLcbblyJtWAJINWK0W6TCUU+J1/L+0mb1mGtYr2+BxMLTxFMEYQQqj79oiWO06/ox+nOCZi3UJUlzs72xMrrc9UoFyxGfP2qkqsFkUkj97bmhENvdzaEtNBE4aqpY1m57aapkp7sOPjpwp7WbGc01P//XML+X4jPB1U1sO89+O631UpC9zn9i9m/F449Xtn95zfn2nU670wg+ntVp7R0CmvPTmcSIFPJXO9qy1TZzgN92QCB7WrhkYgpB38kWtKEWL/6Vjr27xu2MPfCUiceniKY5BTOrAZLBrJr0Mt71qvlfl9PvkC3Vwu93cg1lyAXLSn+kQaCqh1l71ZXe8gSdmo96wyti8D1t8O3/yXn7LXMyh2/viAYqaH3m+i4Vz8yW4pjKG66W4WC2r2YbXSf8s00H62ygzNp1Rzo3vXwnsgqCsCykN++HLloieo94PZDQe59C6FWHb6A+l4sWKzGt0cMPinImpiGjReWOuHwFMEkpuzMapBkIJFJI+3IEfdsz2lkIpU5oIx5SUUf3aAKxpXrMiZEznRjmrnKm8NGTC4lUGl4q/uZ6T61OjLN4k5r//3D4s/CDbDo6DzhLANBFbG1p1Up34H+fMf7ru2ITBpROEmYv0i9Zylh7x7EhvudsFEppVPWWnP3WMgipUQmB5ws9WHhhaVOODxFMJkZbGZVbpv7R1gw25NOBnHWQbigRf3gLUuVI3CvLML1qn3iW6/mGshftlaZFm69NjeDtYXEvIXwZgWZtAhVann/HrViGWkkUTbTelw4nPDW9ferGXxhB7WaOuVTKAy5nbsAser6fOHszgovxfzmnPnGPfNetUa13OxJAFLto2mlm9bY18kKbeveDaqK7bzmYZt2Rhwc4THqeIpgMlOmhaRTB8jxB6gkL1XrrPyPMBc6usaxIcs1l4C0kJrurCyklMq8tG+P6uglNPD5EaaBjDYpoW/3LJYy28+4tbJ7OuY4uGS1Om7NxbkWmIfLRHcqX39RadOazw9z5udm7Db73yuqEZRX80nTsmYhM3eei64p7fzPpNV7BdD03HkLJhF5/oSFixFfuwp278CKd6oqpqmkuvYwhPpIgiM8Rh9PEUxiSvkE3OYgLlmtTAbvtSLvu7konLQQt78BKJ5l2iGF3/9OrvWkXW30rW2O0kCS8xEIVITRUGURdJ86197dana8oAXcZfE+uFTV1XnntZxgrKTk9JFsaOPqI1CWQiWg66pKq2Wp1Zb7+NqIKl9dYEpxBLqUyul7wx2qdHW2+F+5cuTyB7erVZfPr/wC7rwCp3+yqb4XhdFmCxej6RrW3Oa8oAPP8Ts58RTBJCcvSaygj7Do61H+AJd5SAaCJX0HhTkGCKGEhN1VK9t9S1qWapJuC6jasFoR9PfmIlbsInP1URXX/varpUZu34Cykff1AjJXafOtV5WJaM9OtcL4pyvhqgvzBeN49R3w+dXYhtNER2gqtDbZP3ip6UKqamHOXNizu1hJLGwpKh0tpVSCekGLGuP8luyqbq2q15SdBFhZp7Iza0+n1LM1MhCJKhNh9rz5/ZM19f1wrTxFMAQrr6cx4KO9p1flN5Rx/HphopMDTxFMJfLs/y1qZregRf1tO+XK+Q7cn0up5LSRUeUKEErIazpk0jm7v6bDtbeqyKAH7lJx55aphODCxao1YiCIvHcDvPZSwexYKiVy479nq2SWmDl//Wr44R0qPHLtJSOvKlpdq0wnhc7UwRBCObvL2f/X3696Brz1Sv7n0lLKsZzws4vsFa5qehPw1mv5zWds9u3JMwtZlqVMdLtdz33Xu8i1K1Wm8N5d6r3t2o68Z72aFGSVf6FZMW/VEAiqlYdtCgqGih3N991M595WmLsoG2VEkePXCxOdPHiKYArhJJbZNt0bVzllofMiTEpFbBQoEaUJdihH4+7tuVVBbRiOPUHVye/vVQK6LgLfeRDNyChfhHvmCZhfvBSu/lLxgPt6lGCevxDecDmSNR2OOV45Lt0rmpFSGIFTCVKSZ6Jyo+nQNAsuvg7+5Qul96mpK61EbNu8Wwn4fLlVRyqp/CX796p9sj4ad90naXeOK4Vl5c5tmUqZVtiovqwfqWDlafsIxLr7nGijPEHvhYlOGjxFMMUQQiVySbeJKBvpY/+71I+8lL/BbmCubP/KTKAZGbh0DVZHm2o1KSX0JBB9PYhwvRKZhc1O6iLIuvriBiu1EbVt1Vq47H/lZuk1dfC1q9S/7RWNmc09qKTpzGi0q7ST5Hx+NQZ3XwR7W3WtcmanU2q/Qj/IsSfAZTei3b8R65XNOKueuog65xtbXOcUxYuii65FWCaypk4pVzd257i8MQulqO37t7O7JSoLGYob1ZcRzIM6c7OTBk3XsOY1ly1T4oWJTh7GXBGsWrWKUEhVttR1nY0bN+Ztl1Ly4IMPsnXrVoLBICtXrqSlpWWshzW1GeIHWO5HXvR5MKQcxAXZykIIiDYpx6ZhgK4jS5gybPuwCATh9v9EdrTBxm8qhRBugPXfU6uU792Sb6rp64E1lyB1XZkdvnUP3L8xZ36pqQPDhFR/7piqWkj2KcU0Kr6DrFSuqYWLr4Uf3AG7382ukHaoMfb3qvBLUMLXPaPXfXDxdWi6ji8QIB2uV/sbGbXty/8M/3W3iqaaM19FYHV3ZRv4GGq/q7+IrI3A/EVI3eeUj84z7bRaavbv8ylhf+la5Rtad0XOlNbfC2iD1pYaDvakoamuVvW8LnM+L0x08jAuK4KbbrqJcDhcctvWrVs5ePAg3/3ud3n33Xf54Q9/yM033zwew5qyjOYPsNy5NCODVVWrhHpVrVop6LmksXznczZfoWkmsuV9uXaYtm+isBWlZSpbuXSFn7odzn09ShnY6QE1dcruX+hD8PlVXZ9kX+XVPP/lW/Cf/57LuJ3fks2xuF4JWH8A1q7Mjs11TssE6XKAH3O8auaTSmLseDsn5CNRte+6K1R10G/fq5y7dtz+nHm5Fp+mmcsqtlcc2cgtEQxlw0UFTutxzafyC+oiSLstqJFR19uzI29lWI5Knbt2qLHo7Su7j72fZw6a+IxL8/rBeOmllzjrrLMQQvC+972Pvr4+uroqSTzyGAx3hcmxOJf0B1xN2LN/u3Hbh9/cirzhYmXTvvg6ZRbJ9i2Q/oCaZYuCr6IdeTS/WVXTLBTy7qb2Pn++U9b+d00dbLhflWoIVA19o1LCv/+rmpFHoipkVQhlGrvmy6qg33/cVj4KyA6nPfYEsHMxfnA7Vk+2i9sxx8M3N6rFRneXEvhrLgZQXeDWfw/++V8hXK/O425CY680El3I/7gtFyGW6FJK09VbQgihHPX2Kk0IJ6ls8NtXyttacwnWvRuUUvCYFozLimDDhg0AfPSjH2XFihV522KxGE1NTc7fjY2NxGIxGhoaxmNoHodJXuy6OxnJxjFPSejpVoLvjS1wzzqVK5DNTxCZNFy2Ft/3byWz9YV8gV9dqyKHHrhLfT5/kZrd7nsvaxZ5J1eT/xhX8TWBiqFf0AI3fKN8JVQ1AhW+ajuSpaVCWevC6t97duX3Fu7ryS+YV1cP19wMN2Y7gEkJK2+A+25G7tqeMwdFomqct63O93P0dqv/6iIAaJqG9Z0H4e5vq+dkmWqM/b1Z85nMraDshEFpFTmTRTCUM+nNX6SyiEv0ss7Dc+5OW8ZcEaxbt45oNEoikWD9+vXMmTOHY489dtjn2bRpE5s2bQJg48aNecqjEJ/PN+j2ycxY3JuUUkWpDLKCKNxHSkn86GMwdv4ZX8v7qZ8zV+3o3uemu5DJAWLfuQFz24sgJdqB99AXvx9z93bnOCEE4rpbOPiNv4GuTjUT1jQY6CPwk3uw/D5MTeALVVG/+lZE1ondcennVa0bv4+m62/NRUWlksrJ3R2n85K/LX/jDY34Fy7G2LMT6ferGkta1mmb6IL+Poo8uELga3k/xuvK0et//wcJz5iBO64pmhkg9tYrKtQWwO8nsPj9ZHa9i4x3ojXOwNLqoacbEamncWEz3beuxtj+Fr7FHyBy5bfoPLgXKxFDa5xB9LYf0X3fRtKvbFanO/oYGubOg5vuUvcaCCJcXcWcd+banti4GmPn2+qZr7615Hsu9U4HW1FO1d/ZVL2vwRhzRRCNRgGIRCIsX76c7du35ymCaDRKR0eH83dnZ6dzjJsVK1bkrSbcxxTS1NQ06PbJzGjfWyWx3uX2kV+/RjUoCQTp6Ogon6gmpeNItRJxrEwG8a/3kAkEnTLGTU1NyjEsUTHw2cJ46XffUjNm0yCz7UXa9+11bO9yXjNYEjm/hc6srdq6d7XyQVimOlep7F7dp6Jr5i0i0/quEvr1jXDrA2pG/cM7Vcisuzqrzfv/AkNKZXaZ30zmi5fRictR7PMRk1pOCQDU1JH+8r+ofgCWxJrXDJesRvT1QG1YPYOXngUpSb/8HO3t7SoRz7Sw5jUTMyy46JuIlOrrbAZDdHZ2Ftvzy9jrZapTdRKLd5I2LTr27ytbNdT9Tjs7Bw/bnaq/s6l6XwBz5swp+fmYKoJkUs3MqqqqSCaTvPrqq3zuc5/L22fZsmX87ne/4/TTT+fdd9+lurraMwuNJ5WYAwbpaVsuq7koUc3IZKuSGk7Yo7zvZhXmunAx3HSX45SW/kDOeTp/Uc55amTg+7eqksu7d6i2i2vuRNRFlNJJJYvLYtjVPec3q4iagT4nHFP+x23KDJQtsSDqIurzVdercfzgjqxC6FaO29owfP0auOlSdY2+HuU4XrQE7v6pWs1Em7JF7gT2asLXfDRWqAoKne7hegDMdCqnrKSETLp0iG8o5+cYVrLWMMI4Pefu9GRMFUEikeD2228HwDRNzjjjDE444QQeffRRAM4//3xOPPFEtmzZwj//8z8TCARYuXLlWA7Jo5BKhMQgxe0cQVVJopor6xjIVxyppCOEBKiKl7ZSuPvbkDXFsKdVmW8SXWqlsP4K5ELVlpNAUAn87q6c4xZUrP/+99AsE+oiyrQESiHZ9vuvXZmnmLRVNyAuzXZ9W3eFGqfuU0luJcp1a6aBmDFL3aYQSF+2gYzPT/3V6+nqH1BjKSVka8O51puaDjV1QwvkYdjzvTBOj6EYU0Uwa9YsbrvttqLPzz//fOffQgi+9rWvjeUwPAahEiExVHE7ezY6VKKaO+sYyKtfQzCUZ9pwKwXrshuR3/02/Pk11UWrqkZFDPX3Zjtq7XCyZcVla5H3rHMKroGmmrbPW4hlmqoIXzak1UlWW9BSXMo5ez7qIkV1dkRhue7C7l3BEPLYE2HPdliwBK2qGmxFUALNyGDV1WfLe1hw7wbkP984uMAeZrKWN9P3GAwvs9ijIiFRiRloqES1wqzjSmepmqZhXXwtcu0lSvDrOqy5E/7rXhUy6RKEmqYhL7vROa+UUimRt1+DKy9UqwQjA0j4tiqNIH94h6p3JK28toxOQlxhwTZwynWXLdFw6TBm4IGgUlp2yen3WkvO8AfrRufN8j1GgqcIPIbPKJUOGM4sVQRDyIVLgOwqJFwPZYRt3nnTKaS7wYvPr/7r7VEF7b5+dU6p1Tci1t6FyIZyulc9ooQN3t0Nzt0SdNj3JgTi0rUqz+K91pKlpi3Lytvu7kbn4TFSPEXgMWyOxGx0qEJopciVaF4Mb2VzCd7/oVxVTnc8fvb/eY7nCmzwo1VhU9M0rMvWOu0hi0pN37M+1wNi13Yvxt9jVPEUgcdhcSRmo8O5ZlF/hTt/4ji13W0YC0ssOwK40lXPKCVhlWoP6YzFLjBnO8AryBL28BgOniLwmJoUCGhN05zYeVnByqKUg7zQ/APkK4z5i5yWoCMdb55CKbiGuHSt5xPwGFU8ReAxNRlkRl/pysLtAyhn/hFCIFZer+z3eyprCXo44/Ucwx5jiacIPKYkoyo8hzD/iExaOaSzOQWHYx4aaryeY9hjLPEUgccRxQ6JHItKl6MmPIfyFxyBKCoPj9HEUwQeRwy3ySV+9DHIr18zIc0elczWPdONx2TmiPcj8JjGuEwuxs4/Z2v0TEyG6u8wmv0fPDzGG29F4HHkcJlUfC3vx/BCIj08jgieIhgnpJSkTElQH7pd4HTBbVKpnzN3yLLHEwXvXXpMNTxFMA5IKbn5yX3s6EqyuCHI9WfPG7EAmSrCyKlDNEnuYSzepYfHkcZTBONAypTs6ErS2W84f4d8hy88JqowmirKaTBG+116eEwEPEUwDgR1weIGZf9e3BAkqI9McEw0YSSlJGlY3PHMfnbGUxUrJ7fiGGzbRFIqo/0uPTwmAp4iGAeEEFx/9rxRE2wTSRjZq5PtsQF6UiaZbF/2QuVUKNidVU0syaKGIP/2N41F55xoKx4Y/Xfp4TERGDNF0NHRwb333ks8HkcIwYoVK/jkJz+Zt88bb7zBd77zHWbOnAnAySefXNTKcqoghBi1WfvhCqOxmGXbq5PYgIlfE0SrtCLlVEqwp0zJjliSzgGDWNLgmw+/xbWnzUQIMeFWPIWM5rv08JgIjJki0HWdCy+8kJaWFgYGBrjuuus47rjjmDdvXt5+xxxzDNddd91YDWPSU054lxNG5fYfq1m2e3XS0hDkqtPnEPJpeecuJdiDumBRQ5BY0kBKeLe9l5Q5g5BPTKgVj4fHdGDMFEFDQ4PThL6qqoq5c+cSi8WKFIFHeUoJ7+Hubwvk0sKYEa8QKlmdBDRYGPZjmiYLw34CmjpuzdlzWf/EXlrjKT4wq84R+J75xcNjfBkXH0FbWxutra0sWbKkaNs777zDNddcQ0NDAxdeeCHz588veY5NmzaxadMmADZu3EhTU1PZ6/l8vkG3TxYGMiatiVY6+w00TaO2PjrovZXav8qvA0pJHDO7gz+39fL+mbXMmdnE6l+/7fy98dPHDMvElDSsopl/uX2ve/gtXmsbwLDgtfYkt7/Qzq2fORYhBP/+tzNIGha1oQCmaQ7vAU0Spsr3sRDvvqYOY64Ikskkd9xxB1/+8peprq7O29bc3Mx9991HKBRiy5Yt3HbbbXz3u98teZ4VK1awYsUK5++Ojo6y12xqahp0+0gY62iWwkia5ogfy7Jojvjpjceo8s8oe29SyqL9+1xjvPqUGSQN5ZTdd6idtw5209lvYFnd7DvUTsg3dMURt3N4cTTEDUOYmJKGxZsHE44TOWNKXtwdZ+/BNkdJAVT5x+6dDcZ4RCeN5ffxSOLd1+Rjzpw5JT8fU0VgGAZ33HEHZ555JieffHLRdrdiWLp0KQ888ADd3d2Ew+GxHNZhc7h29kqFjWVZrH9iH63xJEuiIa4/e96wTCSVmFTufPYAO7qStDQEaY74kVLSUh8ossOXG/NAxmTL/l4MCT0H+kkaVp5ALySoCxZHQ3Tv78NwCozmKo3a17Esi6RhjbpALnUf9mcBDW55av+EjE7y8BhPxkwRSCm5//77mTt3Lp/+9KdL7hOPx4lEVI/Y7du3Y1kWdXV1YzWkEXM40SyVKg8pJeuf3MeWg31ICdtjyez5tSGdwvZYbGFXLmwzb/wSFkYCqkXiIGO2HcD28bc/c8AR6O7S0YM5tW84ex4DGZPvPLWXXYk0S6IhQj4tL4TUEq1gmSyJhrjhnPmjEglVzsdiX3N+JMB7iRSdA6bzDL1oII/pyJgpgj//+c889dRTLFiwgGuuuQaAL3zhC86S6/zzz+eFF17g0UcfRdd1AoEAl19++YSekR1ONEulyiNlSnZ1pZy2tM31QQIazizZ3sduyO4W1EhKJnIV7nflaUep/YFF9UFeOaBm6a8c7GcgY6JpWpHC6E6arPzVTixAIOlNW86Yq3zKEVxO4LoFdHXAx03nLcz7LGlY6joDhnPObQeLVxmHGwnlvg/bryGEcMJWOwcMIkGNxiqfF53kMa0ZM0XwgQ98gJ/+9KeD7vPxj3+cj3/842M1hFHncKJZKlUeyoQShBgsaghyw1lzHLNFS30AITR2dCU5ZnYHly5rzBNwALESs9oigf7wTprrg9z9yYUkDYtv/GoXSImUcPszB9iVSOUpDCklPSmTWNJECNU73a+BQBl3ejKw4an9XHPGHOc6lmVxqDfNDzYfojWRzhPQhauVoC5oaQhiSUl8wMyuTIQjtO1nVWp1kraUInArWVvQ2+/GPn930qQnZXLHM/u4/ux5eWGrmoA7P7mISFCf0JMQD4+xxMssHiaHk0x05enKQTNYlI0QgtVnzaU7bREOaKQt8oQrQFfS4s9tvUBjXuw+UiJEukjR2ErIFuhdSUn84ACXPNxKb8rKzvIBCa8e6idjSbqTJqseaWVxQ5B7P93Mnc/uZ0cshWFZ9GeU0qgNCHrSSmzv6koCSsmZpkl30uIbv2rNu7dCAW0jpcQ0JaZhEg7paEiWNFZx57MHnBXOFacdxfbOAWJJM291ogl1TXuF01If4M5nD7A9NkBzfZA158xD0zSuOn0OKx/eSWzAZGc8TdoiL2x1STRUkRKYqCUvPDxGA08RjCHDyQOQUnLLU/sdQXb9WXNYFAlgWRZSQl9G4tfgfTNqCPm0vJWJZVl0DJg0VelO+YaBjEnSsLj81NkA3PLkXl5rTyGB7pSV5xcwACyJLtQ41Ooi5ayA1j+5j52d/ZhSrQQGMpJaH/QaYEqlcFafNZe1m/bwenvSOa8GNGcFtF1KYs3Zc9E0Dcuy+PYTe9l6sF/tnDE5blY1V5w6m8t+s9tZ7dz+zAG6kmq1k7Fk3upEiDT3frrZuWdb4HcN9LP+yX2sPWceQV3Q3BBCkMpTlNecORcoVs7lnMuFq5JKQmeH+m54isVjouApgjFkMP9AoSBImZLtsQFHkH31F63oAhZGQuzuTpGxLKJVOt88dzEDPYmsE1kJ1K/+YieJlEk4oHHfpxfxnaf38UpbrttXnV+ZcWxsJSCAuoBGd9bub2ZNP0hIDJjc/vRerjpjLjs6+4mncqrDkEoJACSSJomUScinsbc7v8NYrR8uPeUorvrtbqeUxPon9nLD2XP51uN7eaVtIG//1w71c/sz+3N+jEiA1njSGa8GREIapiXRNUFLfcARyFJKFtUHiQ30I4FXDvTRNZDh3hcOsrMrSXN9kNVnKeFfTjmX8zmUMrHZUV2lHNRDCfiJXEvJY3oyrRXBaM7KSp0roClhJqXMm42WEgRBXdBcH6Qr2Y+U0J1SNvNYUs2YBWBZ8LkHXwLghNnV3HDOfLrTFvHsjDmRsvjSz3a6wjQVbiWQN2ZgQVjnzzFJxlIH2fH+JvDKoQFueWJPnhIodY4rftNKc0OIlvoAWw7llIEUAh8W8yMBYgPKJt/aleLbT+wrUgL2uVrjKb732cUIIQhocPNT++g50A9IQroABM6rEmo11JORhAMa15wxhy/+v+1kLKXUrvzdbsf30Ns2QMqUCEGRcrYzrC3LcpSxe5uUMt9nMmCyoytVsrBeJQJ+otdS8ph+TFtFMJrL/XImoFue2s+ueJrmhhCrz5rrzFwTKdOJXAFbEGisOUeZYXZ1JTEsSV/GwsgKZglIpCOobUFU5xf4NJz9CpXAULzVkeEDTUH29qTpy0hMK7dikMCr7ekhz9GVtIgf6Of42dXU+aAnu1roSUv+8Wc5f4FPQHNDiNauYiVgszDsJ2nYfhLB9WfNJWlYbHxqH68cGoBMLvt4R+cAX/n5DrpTFpGgxo/+ajEnzK5meyxJb9qia8Cdqazea6HzPqDhJMiZlqQ/Y+HXlJPZ3mY77G2fyc54sT8GKhfwXi0lj4nGtFUEw1nuV3IueyZpWRaJlElQF7nQSAFpC4Iil5VrSYrCFjVNY+0585zZ6apHWp3ZaTggeF9jFa8c6s+boaYlRIK6Ewtv48va0UsVbbCjfkBtf6Mj36Rjyy5p5X1MjV8ohzHKhFTtg94Mju9gV6yffoOyGBJM02BeJEgsWVoZ7Iin+fLPduDTlNmqJVrF1acfxZ5EcWP7ueEgr2VXFvGURXfa4oZz5jOQMbnk4VZnpaQDJxxV7WROX3n6HKSUTgjru539dCVzNxutUk5mt8Me1Pu54Zz5ZVeRlQp4r5aSx0Rj2iqCwoiacsv9SghoYEklYHvSFlf8upXF0VxEiy0U3CWbo1V6ybBFOypJSsGSaCib+BTkm2ccRZVfp7Y+SkdHB3c8s1+Fg0b8LKwPImUSUwLSoqUhxNVnzOHiX+50Zuc2fk3wwZlVvHawv6SS8An4i5khXm9LFm2//9MLufOFdvbE1Zi+sWwG1/1+N4ms6Sk+9OKB1niK286fz0WPvFdyux2NZFhqpfHy/j42PvkeGTMnqDXg+NlVStO5702ov9MWeVqsLihYdfJsLMtiw5P72dk1gESgCzAsSSKl9vVpEA7oTsIbUCTYB4saswV80rBKbi/c1zMHeUwUpq0icP9o73hmX9nlPgxepsCOadeFQAKmhYpsieciWnJx7fmCZbCwRff47nz2AJf9Zjct9QFu/ssooBKvDKnyBxpCOpYEDcmixhrWnj2H/rRRpATqg4LbP7aAu547UFIJANQGNS5aNpOVv96T97kO3PzEe7zXbdJvQudAP5c8spuqYcqyRBqu+v17+LWcP8K9QimF2/HtE/CDC1oI+TS++P925O13x3MH8WkaO2JJEBo+Yd+l4Krf7sKwIJEy867pHn44qHPXJ5vz3svhzNztMh6VVIz18JgITFtFAErYVvn1QZf7g5YpyH62+qy5TjKYKaUT417K53DFaUeRMuWgSqCwfIRtdupOGnzhv15mYTiQ5wuwwysB4gf6WPfH99jemSw8LaYlufjh3YP6ETIZi2t/v6focxN4q6tYfQwM0ycBynntE+o/26wkUDN9K/t/DSjl47bfmSL/4ru6UiCUcmys8vHDv1oM4EQtuQkHNXyawLRUtrQQqiZSuRVapZTyE3h4THSmtSKwGezHXu6HXfiZnTRmrxAKVw9Jw+KOZ/ezLRsBY0f92Mfb+7sLz6nVgyAxYMfRQ0dfBsuy0Clt/5fA9o4kPSU2losectM3TpWgDQnhAHRnzUkSCPkgaaj7EmWWCcc0BelPG9SHfMox3KnKW/s0QUs0iESonIGoWnGlTKk+j0m6shFEfg2+95lmNE1zMrPVd2D4wQKFq0XPEewxGfEUwRCU+2G7M3u/8/Q+dsfTtDQEuOqMuXkCJb+nr+WEae7oSjGQMbn9mQO0diVpjoa4/syjWP/UAbYd6FNRQjKJEKJI4M8LB1kYkWw7OFDSpDJewnykdBf4FPpck3Z71eJ2UAO82pbkq79opdYHP7qgmVufa2NXV5JFDSGktLJRYCGuO1OV6Hi3s58F4QDf/cQCVj6yi+60pDqr9N2VWK84dTbxpEUkqKNp5ctxF67WSoWLeo5gj8mGkFJOyrXr/v37y24b7Xri5XwEScPiO0/tY0s2O9avQV1Qz4s+ShoWKx/eSWe/oZK1stbp42dVIYXGlv19jpCr8wt6MrnXsXR2FZom2N45QG9aYkhlTqkNKDNG2sSJmpnM6KjJfykXa0NQsLBehZwmChSHBtRX6Y7zPeaKnLr/M4u47tHdTg5EOKgjpEkirZ5hXVCjN63yJ/yacBR0JKjxn3+9xFktuIX+QMbktqf3syuhSlNcefocVj2iGgE1Vvu47zMtJXs6TNX69t59TT6OSD+CicbhJpCVK+0M0BrP2eIzlrJP74jloo/yevrWB7jy9DnOtVc9vDNvRu9WAuGgzg1nz+XWZw6q6p1+u/onxFOSVw8l8VmTUocXYYvvSFArKn+RNiTXnHEUAxmTr/1qd95xFup5+zXBgojKKra57rE9mK7n050yqc7KaEOqiCSfgIYqnd5UToEksmGo4QDZnI4UzQ0BLKkyn20H9/ZYrsaS/X/PDOQxWZk2isBtonEXJRvJeRZFAiyqDxJ3mWgEqnroUP13LcvCGESQ+zXoGDDZHhvIi3G3yViypDN1pAwVwTOWJFIW74v6eCeWsxH1mfCln+8cdFDVPlnU5jKeFfQ1es5U1l/wGKt0uPdTi7jzuQO8vF+VpggHBH4h+fbj77H1kFptJZJGnoNdoMqEF9Z8GmntITvsdKR1jDw8hsu0UQR5tXySuaJkw6066T5PbGCAE2dXsfSoGl7NJnodN6uKNWfPLZMbkCuvnDIloox0E6gaPit/1YqFMmVU+3M1gUZCCEhRWq7ai57hZiePJm4lYDNUWH4iDa+3FyecGRKqdEFYFyWfncgm+t2QDdNNGhb3vXiIVY+05ilft772a4LjZlfnveOR5gPYk4utB/pwBxJ4ysBjvBhzRbBt2zYefPBBLMvivPPO44ILLsjbnslkuOeee9i5cyd1dXVcfvnlzJw5c9THUVjLZ1cFyWPlagItigSIDajZ4uttA/zX3yxxCYXi2ZwTNfTMfnbGVc1/aUl607LkDFySL4ylBMscuRIAKA4qzb/uZI12LDdslaBWemt3Br7+i52ccFQ1N5ytVog7u1JFK7AaHyQt9R4+lFX0h7OaLIc9uXAHEnj1hzzGk9H7NpfAsiweeOABrr/+eu666y6effZZ9u7dm7fP448/Tk1NDXfffTef+tSn+N//+3+PyViEEKw5Zx5Lj6pRpR2iQ9t03aGj9o9TCME1Z87FrzmVz5xaQVX+/Bh0uxz0zU/uY+XDO9l2sE+dK5ZkZzyFIVVl0LJjBidMtHccIoEmqxLQgcYqnXCgsq9zJKBRl50CZSzJ9liSgYzplO5oCOWfp9+A2qCOIWF3PE132ipq05k08j9zb7PPXQ67r7NfE/g1z9/gMf6M6Ypg+/btzJ49m1mzZgFw2mmnsXnzZubNy2VbvvTSS/zt3/4tAKeccgo/+tGPnDowo427lk8lNt1yoaNVfp0TjlLFzSxLcuVvdxXVKXJ8CZ0D9KQtJzqlIaR+9HYzmbk1Oq+WMGuA0tI12RLSk1RGjwtCwOwaHb+A19rTZbOmfQIiIV2FmpqGUym1a8DkG7/cgU/XWNwQ5L7PtHDrk++xLZvRLAS01IfYRQpTSq74TavzvqF0CCnkvgOtiVaaI/6ydazsvs6ej8DjSDGmiiAWi9HY2Oj83djYyLvvvlt2H13Xqa6upqenh3A4nLffpk2b2LRpEwAbN26kqamp7HV9Pl/J7fbMranMD83e7v4h3vk3TUWfSSkJBNrQRIrejEnGgtZEhtr6qJP1OpAxaU20Estm/fp1QW3AhyYkgUCAjZ8+hpSpnJwf/48XC8vmACoqpjujHMeGla8MjqRTdzwolzBXCkPCGx3FhY4Kn9GH59Uhhc7Ozj6ObqpBI+WErHanJWCi6wYzZszg7v81k6t/+QZb9iYACAb9PPgPx/LV//82OvoyzvsGaE2oEFJN00p+B9p71djc20p91yYb5X5nk52pel+DMWmcxStWrGDFihXO34PF+ZaKAx6qVvxQZan7XPslUiZvH+yhc8DArwmiVRrNET89XZ10ZrOKARaE/XT0pZFS2ZmRJp0DFm8f6mF/Wwchn4pVXzanhu2xJKZp0ZeWjgC0rQOWVSzUprISgMqUgF2Sohy6lu9ofum9HiyyvhCrp+hYn4BFYR+98Rh9QnDFKTNZ+XAvsQGTP7f10t8dpznix7IsZz/A+aw54neOBfVdaY74nX3sbVOlMc1UjbefqvcFRyiPIBqN0tnZ6fzd2dlJNBotuU9jYyOmadLf309dXd2oj2WoWvGVlKV2h46qMtLKzHD16UcR8mlOs3n7x+3ujdtSH8BCILK9eAOamjECrD5zDuuf2Merh1QcfDig4dcFCyIBXm8bOKJRPBMZC9UFrTcbR+uuXVTnh6OjIbYcyrnH3cqlN2U67TZt6kI6V50xN8/xvyQaYkdXylV9NOs/yCYLDhZCaocO19ZH6Y3HnG1HojGN1xrTYzDGVBEsXryYAwcO0NbWRjQa5bnnnuOf//mf8/b58Ic/zBNPPMH73vc+XnjhBT74wQ+OyRd1qBowlZSlTpmSHbFsGemQxvxwiNauJHc9d4ArT59T4setms0kUib3/ukgu2MpmhtCXHvGUax7Yi+vHeoHBMfNqmZnV07gd6ctjpsZxLAsJ4FpMCo1owzH3DJZSJoQCcCihio0AVsPqmiu/gwITaM+pNGbsoqUaSYbBWQ/E78m8spP29g1pEI+Le/99xzoz5sslMoohlyRvD7Xd7rcd3GshPVUWYF4jB1jqgh0XeerX/0qGzZswLIsPvKRjzB//nz++7//m8WLF7Ns2TLOPfdc7rnnHi677DJqa2u5/PLLx2QsQ9WAqaQstV9IDMvu7yvYFR8gnpK82zngtKOE3I/b3ZDeXWdo3RP72XbIzoKVvHqoH6sguezNthSFEfWRAEVlFqBy4W4C1VpxYtV4EAlo9KStQU05h+P3MCz1TPbEk87KgOx5Xj2UC8ksdW4hBIaUNIQ07vpkM/UhX17Cn51ZvDiqhGdAU9VlBSraKDZgsj2WzGt3WYkQL/VdrERYH66i8FpjegzFmPsIli5dytKlS/M++1//6385/w4EAlx55ZVjPQxg6JLC9uzt+rPnZcsM5DuINzy1n+5sExMhLWS2RGY8aXH703u5+sx5eVUsk4blNKLxaxCt0lnUEGRnLFcfSIWhKr9A2A8Wgt6MzFMCGnD8rCBf+fBs1jy6m+5BuoANxZFQAgCGNbgSAPKys0sphCotG89fYtuccJB3XKW3j23083ZMaQYN+MCMEG+257YfPzNEwK+zM2v2cSsBKSXrn9jHloN9yokfy1Wd1UT+OC2pJgg3P7l/WDPuwu/iUMJ6JLN6ryKqx1BMGmfxeGHP4gt/cEnDorUrqWrnC1jUEGJPtnymBF45lGTVIyqscPVZc0mZEr+QLAj7MU2T5oYQ3zxTVSa9+cm9bDuoEts+NLMKgcXOrjTdKQuzhJizgNfaUvzLb3YPe8Z8wqwQ0rJ4ZZDew+MRgdQ3iPIqNFmVG8vAIJokqMFfzAixK55mYX0ATeQ6GliQpwQiQY1rz5pLlV8nbSlBnkiZhAMamqZMQK3xJFLa7zq/6mx3UjUFkiiHfk9m5DPuoYT1SGb1XkVUj6HwFEEBpX5wQR3ueGY/PSnTKTFww1lzuOXp/fRk+wtIaRecS/KtP+5ldzyJaclsWKKyXd/xzD7WfGQBV50xl5UP7yQ2YLIrofwGskS5aTeH6zC+7JTZ3PtiG1BeERxpX3S5+64W0F8wOB9QHRBYlsxz9L7WnqQuqLOwIYQuYEtW0ZaiL21x6a9V7sd1Z87hq7/YSSJlEgnqPPhXiwnqyl+wPZZUdalc5SSuOmMul/xqB11Jy5kQhAPaiGfcQwnrkc7qvdaYHoPhKYICSv3gUqZkZzxFxlKNza85Yw66rjtJQFJK7nzuADtjSTKWZNvB/pLnfrdzwPmhNzeEgCSL6oPs6koRT5rOzNxOfMqYlqNIyjHUbH4gbfDKgdLjmcj4NPjOx+Zx6W/zM9EtKPNMRLb+Uz9+TTmCBVnzTXYPXahS370Z6QQDdAyYJFImUqo2lt1pi/qQr6xQDvk0jm6sylMSmjY6xecq6Yfszeo9xgJPEbiwnXGrz5qb12WssNewHSHibpt4/Vlz+dbje11O4BLnR1W2vOWpA7xyQGUmCGnREg1CF7TUB/naspk0VekkDYsv/r+dzrH1QcHsWj87ujJkLOn02tWFqqLZa5RWCIWCdLJgWfD9lw4VKbpy1qEqn8BIqwY2mWzeRW1AIKSkO6P8L80NVezpTlEb1BCochJNVTqRoO6sCOwyFeWEcjmBPB4zbm9W7zFWeIogy2DOuEpmY2kL9nTnSkXoKEHUZ0gnqcmva/RkVIEx29TTmshw32dUy8o7ntnP9Zveo6UhyEDazDOZpDOStztz5p3jZlWxJ56iK2Xh82nUCYvuIepSC1RiW+8InM2jhQ7UBDR0TbCwPkhV0M/r+xNYUvkTLOC19gzhQGVVVzUkx80MsSuepCetjk9mlJHfr6n/v9o24Ky46oJatsyH4HufaXb6SA9VTM6Lx/eYiniKIMtQzrihZmNuu/KiSIBrso7hpGFx2zP72R1PszgaVPbkaMjxLdgrDNv81NlvIKXEMPIt54XRPru6BgBVpKw3LQlWIJMkI1cCApWsNZTSsfctZ7ayUEJVSolfg1s/cyyX/d9tvHIov+NaJaJWAGiqGX1fJrdqMKS6RkOVDlIiya2kupIWO7pSrHtiL7uySX52D+lyePH4HlMVTxFkGQ1nXKlVQ3VA48aPzM/7vFSBMcf8JMG0JP1GdgYfEPSWsIkn0qAjsyUTJDUBiryudX5YVB9kX0+GWLa0cq0OAyZOqYXh0lCloypiD569MJTvQpLryLY9liSeNNiTKC6+J4d4DX5NmYI0gVPR1caXbR1qWdCbtpSjf1Y1AovWRIYFdT6n61hPqt/pFWGHDheuDrx4fI+piqcIsoyGM24wu7KdcBTQJGmruMKkff1EyuSK37RiSFQ5ZCFwC12/JjAsiS6UQ9SWe4saQioE1WVG6cnA7kSa9zdV032gD0NWVs5aQ523KAELFRnlvkNdqCS1AamSu3yaivcfMEtHOlUDela52Zu7kxa3PPYOi6MhXjmosq0/0BTCJ3C6hLmJVuncvGI+4aDOXc8fZGeX6vFgmSavtiWdcRw/q4pLT53Dlb/dhSEhGtK45sw5BHVBImXy78/td2VuCyzL4iu/2pUXQWQ3IwrqYsSTBQ+PiYqnCFyMlTNOSsmGJ/Y6NYp8muZkqxYqg0hQd+rbtNQHQAi2HejPlubGyZStC2osqg/yelvWROTTuf+zzax8eKfTsB1UhM0/HBflpf19hcMiHBDoQtCbsRwHqy6UAK8PChIpmacMJEoRVfsEvWkLE3XMkhlV+HSNHbEkPSmTnkHMT2kBNVISzjpsEymVTPfS3m7+47PNBHXhOOpXPdyqnotrXEJAc0OI+io/IZ/GDVnlHdDglqf2U+NXze6vPXMO1QH19XaeZ7aEyC1PHWB7bIDubGVYAXxwZhUpU+ZFECVSJvf96VCeKWisI3c8H4THkcBTBONA0rDYdrDPNfu0oKu0aaFwZWIfb4eovnJQKYUl0RBXnj6HVY+0qlDIziQ9aYsljVW8cjBXWgHgmkf3Fs3uj59VxU0fmUfagjue3c/OWIpFDUGEVGaT5voA73QMFIVq1vpVYTe3HX5XPMWdn1hEUBdc8nAriVS+Q2NJvcaOuOV0XktkoCEEN69YwMpHdgOQMSVX/qYVKTR0IWiJBjGziQB1AcH7mqrY2ZViUX0QISWrHml1hLPti9keUyU/Em0D3PHcQacV6eqz5pJImdzz/AFWPdLqlPvwa9AQ1LAk7I4nufdPB4kENBJpi0hQJ6iLkvWjxsoc5PkgPI4UniIYN5TVXAANIX1Q00LhysQdoqrXROhLdDmfLYmG2N45QHfK5OJftRIJavz4r1tIGhZX/nYX8ZQkY0mnH7EQcNysGtaco+Lfq3ScWbVdH6k7bRHQ4NJHWgFlCqr1C3yaYGGDMt+41UNX0uLrv9iZtb/ntgiUEG/rz19ZaCib/X+81ObM9AG6UhIh1IxcxiR2EzhdE1x1+hwnw/vK3+4iNqBm87YyLdeKNKjjVIztyRafs8t9LG4IsuqUo5zztcbTfO+zLaQtnPIi42kK8nwQHkcKTxGMA0FdUBPQSCRNwkGNez/T7Ahy20E51MzPLn2xM95Kc32AG87OzXZv/MN7xNqULT2RsshIQbQ6wJJoKBuFIzjhqGquPO2ovFpINrbicZfXaKkP0BINQSyJYUmElDRHQ9xwlp2Jmz/rNyRsPdiPhhK0H5oZ4qLls7Esi5W/3pO7FqpPQMZSbR+r/YLutERHKQ0pBD5NoyUaVKUxDg3Qm1arISTs6FJd4WxBbgtnuxWpu1BcUBd5qzGlhFVp6avOmOvkg7hLTVf5dapdz2Y8k7g8H4THkcJTBONA2soVK9M14USjVGIGsG3GlmWx9UAfGUuSGDBIGpZTK2dfdy6/IBzUqPMrASiERm1Ao7khxOoz55CRYlCBVjgjvffTzSQNi6/9fAeGREXYSMGDf7WYbz+xjz1xpSRs85Hq8WXPpnUu+/Xuol69di3QaJXGokiAXfEUINF1gdA1jo6GnKZAScNySnHsiCXpTprZGb3gBxfkVwuFwVqRqtWYTxNFVUZhcGE/nklcXvawx5HCUwTjgJ1j4G5wUokZwG0zXhj2k4vjkfnnbgyxvTPJgvoga8+ew8anD2RNIaqNpoin2PDUfide3u61WyhwCmek7gxqVbhH7afrOjd9ZB7rn9jHjlg/kaBKDLMFtSVhZ1e+n8JGAMfNrubq04/izmcP0JPOma26Bkx2dqUQQjgrF/u5LaoPOtnYIMu2eCwU3CGfxglHVbMjlmRxNFSkBEodcySZSGPxmD54imAcKDXTKyxbUcoM4FYWUko+NKuaPd0ZmiP+PCHtPrd9TGzAxCeUKWRRJMCurhSdAwZIyaHeND98qY2dLsVgC9/CcRYKUtvcIqVkVyJFPCWJVmlsPH8h1/5+lyrGhsSSuRh8HagN6mhIWqIh1pytqrNuO9jv+C8+PC/CO229LKpX3dsK7y2gwc1P7nWUablGMKWe/Q3eLNvDY1A8RTBOFM70KjEDFM7QV581l7qGxry2h4XnthWMlBIrG26paSoKh5gKj/zGr1rzEr7cqxF3zkNQzxekdoim40NwjW1GtY+jG6vU7N0x+agVgAn4NMEdn1jkzOSltDCyKwZTwk0fO5obHn6T1q4ktzy1L0852WO74Zz5hyXQvVm2h8fgjIki+MlPfsLLL7+Mz+dj1qxZrFy5kpqamqL9Vq1aRSgUQtM0dF1n48aNYzGcCUs5AeWOJS9UFoVtD0ud005Mu/I3u+gcMBAizb2fbqY7ZXLRL1UhO4laLRSuRsqFMIZ8uUY7bh+CUhyiaPZ+y1P7IJbClCr6pyUa5N4XDjqrkCtOO8qJGNKF8qPsSqRKtggd6nl5eHiMjDFRBMcddxx///d/j67rPPTQQ/z85z/nH//xH0vue9NNNxEOh8diGJOSUoK4UjOIjZ2Ytjhb1dQ2pQR1QX0oW2kzoHHfZ1qo8ut5s+vBfBelfAjlViZupZC2VOtHO+fB3vfEo6odU09Dld+LmPHwOEKMiSI4/vjjnX/bTek9KmM0Y8ndjddtM8uDf7W4bC0dGDyEcThRLXnmKiG5+ZmD9KQsfAIWhv0ENNXkxR7faNX09/DwGD5j7iN4/PHHOe2008pu37BhAwAf/ehHWbFiRdn9Nm3axKZNmwDYuHEjTU1NZff1+XyDbp/ISCk5ZnYHf27r5f0za5k7a0aeUKzk3qSUXPfIW845Nn76mLxzzBxiDHf+TRNJwyobmTOce7Edy62JVjKWJKALdndn+NqvdqFrwhmfz+djxowZh32ticxk/j4OhndfUwchCwO9K2TdunXE4/Gizz//+c+zfPlyAH72s5+xY8cOrr766pICJRaLEY1GSSQSrF+/nq985Ssce+yxFV1///79Zbc1NTXR0dFR2Y1MQAarN1PJvdnx9539Bo3VPu77TMuwzUsjxW3ismsmqVpEqryDHZFqj2/e7JmT+p0NxmT/PpZjMtyXlBLTBD0b+FAJk+G+Dpc5c+aU/PywVwRr164ddPsTTzzByy+/zI033lj2BUSjUQAikQjLly9n+/btFSuCqcxInaITIUO1VHIaqLpGO2JJLAl6toSD5w+YOByO4JyoSCnZ/EwfiS6TSIPO8jNqJv09jRVjYhratm0bv/zlL/nWt75FMBgsuU8ymURKSVVVFclkkldffZXPfe5zYzGcacdEyFAt51i+ocCJXOn4CldJXpXO0WeqCU7ThESXSXJA5bybJvi8gPmSjMljeeCBBzAMg3Xr1gFw9NFHc9FFFxGLxfj+97/P6tWrSSQS3H777QCYpskZZ5zBCSecMBbDmZYc6VDLSnr7hiq0VhVGUq0+a66Tz+BV6Rw9pprg1HWINOiAUmy6fqRHNHEZk9d89913l/w8Go2yevVqAGbNmsVtt902Fpf3mCCMljIqNDN1py2vSucYMNUEpxCC5WfUTBlT11gyifW9x3Sh0MwUDmhH3AcyFZmKglMIMalXNeOF94g8JjylzExH2gcyVfEE5/TEe+Uek4JStZo8c9CRQ0pJJmNlW6h672Gy4ykCDw+PYWFHF/UkeqmLiEkfXeThKQIPjynLWOUEuKOLLEvkRRdNpTyE6YSnCCYwXqy8x+EyVE7ASAS2HV2kaZK6iEDTJIYBQlhsfrqPRNyiPqpz0pm1Zc/tKYyJhacIJijlykF7TC3GQiBKKUkmLeJdJqkSOQEjTRyzo4vq6xvp6urgpWf7iccMpAXpbNfU9oMmhiHx+0uXWT/c63sKZGzwFMEEZTSrkHpMLGxhpmmSl57tdwTistOrsSwxIiEnpeTFp3tpP2giJWgahAtyAgoTxwxD1X4aznWFEPj9GpYlSHSZpJKVj/FwE9emWubzRMJTBBOUiVAvyGP0cQuzcL1OossglQSJyean++hOWCMy5RiGJB5TSgDAF4Clp1TnHeNOHAs36Gx5vo/ueOnrFo69cAz2uaQ0kBKMjGp61DRTx1dm4nK4iWtTLfN5IuE9xgmKFys/NXELM4lJpF6nO2HlKYXDNeVIKdn6Qj9Gxv0hRYLWnTgmpeSJ3/Y4wjWTsbAsQSBAXr+KUmOwOfGUaqSUSCnZ8lwf3Qm1wiiH+/pCWKRSkkBAluyP4eZIZT5PB3OUpwgmMF6s/NRDCIu6OoFlSepd5qBCM9HhmHLs/SzLfT2y5y8ch0ockzInXOsigj/+pod0CgJBOPP8WkIhVSwwlZLEY/mKylYO8S4Ty5IIIJMBaUF33MoKz9JCVAiBplk89qvc9T762fCgykAIwbLTq0mnIRAYH6E8XcxRniLw8BgnLMti08M5wffh06rQNC0rpMuXd8gz5dRrbH2hv6Rg0jRJuF7HkspxCxCOaFiWhZSlmwy5Z+fptMkfHu4FIJ2CPzzciz8ADY06ibiFlBAMQV1EkMlYpNOmyyGdIxDKRRVtfqY/awbTWHpqDT5fbnWbTqvr2NdLpyEUKv3spJQYhnTMWOEGnaWnVOedbyyYLuaoKXhLHh4Tk0LBl8mIvJl/qfIOtlniw6dVkcmolcOTv+vNE0y6nhOS8ZjqCa3pAmlJOtosHvtlD40zNT58Wm1JwSmEQNclr27uLxpzJg0dh9QqQ9PgnE/W8MxjfWz6VQ/BUC91EY1Utg+1OhecdX4doZCWJ0TTKZM//raHepfyCgSUQrQVo6bJokxlWwFsfaGfeJdJJiWxLEglDR7/dTcNjb4xnaVPtUJ85fAUgYfHOCClRNNknuALBIY+xja9yGxXt0iDTtglmDRN8qeneol1WpiZvKNd54GONqtIENvXMAz1X3dcLSM0XQl00wB/gDyfg2XmlFkqKTntvCBPP5rzSzTN0gll64tLKQnXa0hpkklDakCSKPBDfPSzYVIppYSe/F1v3irHsiw2P9NPvMvASOMoI7Ld7dIpiMcMZ5bujsYaafSVzVQsxFcKTxF4eIwxdkinMpEITji5lmBQzcwNQ5YVMPaM2m166Y5bnPOJuuwsXplz2g9aRccKDfx+NaOXgCAniN2C88Wne+k4pGb0Pr861spO8Jtm6Sw/o4qXnxtwTFE+f74ZyO/X+NgFYQYGTCwTQlWQTFq88mIfPQlJuEHnnE/UsfVPA3R3FfshPvrZMH6/RnfcchzoyaSFpkm2PNtHR7t07idYJYhEdOJxg3Q2XDVSr2bphUoTAfUNPpafUT2o36ESR/B0KMQ3Zrf305/+lD/84Q+Ew2EAvvCFL7B06dKi/bZt28aDDz6IZVmcd955XHDBBWM1JA+PccMtYNJpk/ZDJtKCTFo65pmhMn+dGTWWI9zCDbpzvJSSrc8PlL6+pWbMQkDjDIGu63THc+YNKSWplKQr5nYuC9wriZ6EBagMYXumvfnpPme7pilHtJSSZx7rc1YKbkzLwLIsTjy5CtNUfpLHH1HnSKcglZIEg1lfhjSxTMkfHu5xwl9tfD44Y0UNgQC8/GySRMIkHNE48dRqdZ0SSrPtgMHmp/s46azSGc7TxRFcCWOq5z71qU/x2c9+tux2y7J44IEHWLNmDY2NjaxevZply5Yxb968sRyWh8eoUG42mZcr0KCjaynHeQvqe59KaUVROKXCRcP1OnV10NGmhLp0SUjDkMTjOft86TFCrF1y/gW2Y1pVDd36wgCJuEnGJbzTqXzpG2nQstcT+HwCw4CE63qBoMZTv+8hHBYllQBAJgWP/VIJfiGgobFge9rktc1pEgmLSMRHvMsoUgJqP3jq0V6Q6t8I6DJMnvhtD/VRH8tOrybcoCNRKwJ7PIl4eQdvJY7g8QgdnQjhqUd0wbN9+3Zmz57NrFmzADjttNPYvHmzpwg8JjyFs0l3VnBhroAgJzwtC574bS9CE04UjtsJac/U7WgcW7CpbflhmVuf788T5I0zVARSZ7uVF0IqpTqvrsOLT/c5WceD0dAEliV5/DfdRBp0PnxqjYpKimjELYtIvUZPt0U6hWO+GfqZQVenWqF0d0syKXjydzkHdTcm9Q0aHW1WaWXgVjbZ5DUjo561aaoLWKZ07t32qZRz8A7lCB6PFcNEWZWMqSL4/e9/z1NPPUVLSwtf/OIXqa2tzdsei8VobMxNERobG3n33XdLnmvTpk1s2rQJgI0bN9LU1FT2uj6fb9Dtk5mpem+T7b4yGYuehIre0TTJtj8ZdHWmaJwR4txPzGLGLIPO9iTRGUEsy+LAe0lHuKXTgJRU1+h89u/mE6rSHVPPH35zkI62AQQa1TXQOCOIZUkO7BtAIJg5q4pZs5rIZCzaDyXyxtTfB+F6P5ZVPD2PhBvQdZ1EV2JIJQDQ1QGgJGr7AZNND3ej6xqZjAQJvoCf+kbJoX3pYT03KcEfCPKZv2niZ/+/vc7nmgYzZ1XxkY/P4g+/OUhnexLTlGr2PwQNjUGi0Sjd8Z68/UNVGh/99AICgdKaQErJik9FAeXrsAWw/V0sfMf19Y34/RU22q6Q8bhGJYxIEaxbt454PF70+ec//3nOP/98Pve5zwHw3//93/zXf/0XK1euPOxrrVixghUrVjh/d3R0lN23qalp0O2Tmal6b5PtvqRUlTctS1AbFnS09ZNKgmX109bWyfEn+TCMGrY8r5ymbtkbCIDQBHURQW9fF339SgAZhqT9UD/JAUkwBKevCBMIqBXGX3zYl3VaCjo7O7NNYfLHVFsniHWUttE88eh+uuNWWRPOUKjZd26ZcXBv5SfSfWrmHWtXK6OD+5I8+dhe5czO5Pb54FIf7e0xOtoGimoX+QPKVBVrt7IrnNy2A/uSPPbIexhG/gOpCwsSiVhZ/4DtwI805FdKtb+L7ndcFxHE451jsiKoiwjM7Peoq6tjyAzrkTBnzpySn49IEaxdu7ai/c477zxuvfXWos+j0SidnZ3O352dnUSj0ZEMycNj2ByOjdYdVujOCraTrTRNOXTdBdmEpmrwLDu9CsPQ8rJjpZRYlkVdWGBJSTjiQ9dNNj+dJBE3qY/mx8vrujKxdLQpJ3LTTI3lZ1bzp6d66GzLjTMQFNSGJe0Hi30J9Y0Qz/78/AHQNEEqWZmZZzjoOsS78q/f2Q7+oGssDVrWNCMdIS+E2icS0Vl+ZjWmCb//RQ8UDlFCrNPMy56eMcvH8jOry75Pw5BOfkTHodKVUscjdNTOlt78dB+JLoOXnu0/IuahMVM9XV1dzr9ffPFF5s+fX7TP4sWLOXDgAG1tbRiGwXPPPceyZcvGakgeHkXYNto//qabzc/05TljK8UWGGd/vJZEl8WmX/Xw2K+6IVtLyP5NB4KCpadW8/JzAzz1+9z1LMvixaf7eOyXPXS0W6ST0H7I4Pc/76PtoFIkdry8PeaXnu2np9u+CZUnkExm6I7nxqX5oC4CsfbS4453QtNMwfkX1HLOJ2rIpEemBBqa4OSz85MjIlEwTLCM4v1tm78QEI9bvPhUH5mMhdBs5QjhOsFJZ9Wg68p8ppWRjwKlTIIhmDlb56SzisNG7faadotN+1UP9srtVZhtujMMeVjfkcGwLEF3wiKVdPk7xpkx8xE89NBD7Nq1CyEEM2bM4KKLLgKUX+D73/8+q1evRtd1vvrVr7JhwwYsy+IjH/lISYXh4TFWjGZJZMsSeZnDLz3dz/Kzanh1s0l72wD1DcpW7Z6JZjIWW5/vp63EjN0tb8JhDSEsMhm14mg/aOQ5hJHw+CP59hRpQucQjtyONqkiiLryHcyHQ1cH/OnJfKN+Ijb0cVIqpdB+yOTxR3qJNAhS2ajYzk6ZLU6n7qNpll7S2a3p8OHTap38ipLZyc/3035IaaSGJpVnkU6r1dBQGcOWZTfdKV6djZRKs5fHMrpozBTBZZddVvLzaDTK6tWrnb+XLl1aMr/Aw2M8GE4JAfcPsZQCcZdMADXLtSzBeZ+czf79HY69300mYxWZTQoJBCHRreoUGRkqFtiVTlw72sy88NYjiWEUKC+ZFcLPJEl0mdRGBL5sopyb+gZfXvkM+10JYfHSswN52clAnvnMyJA18ZUek1qx9TvmtcJsZts3cbh1jyoxQY11dNEUz5fz8BicSipaqrh/yWubVey9HS5aqECE0Djv07W8+FQvne2qRMOW5/sJBFQkkH1c0ywfHYdUvPzTj/Y55RmEpoSLW8jpPkDgZNKOBeOhBAIBOPXcIM9uSmGUMBOVQwjY9kKSWKeBkYFMRmIWHN/QCMeflHM4OFnGMbVqcj9PoRXf72D6UkqVROf2sRRmM7cfVANqmjV4e87B73Pw7OWxLn7nKQKPaY1tby830zJNk8d+lR+WKKWBZRXP4qSUbHk+SW+vQGgqnj0RN9FE0vkBW5byE/zxN92kkvlCSlrKYds4UxDrsJCWUiamgfLmVSCwVbbvaD2d0SOdhid/N/yQJctCzeazyrJQCYDKTXjsl33MOErn5DNrs815jJJd0zQNzILn0zhDK9lEx53TYeN2QhuGzJbgVtts+/5wBHSl5p6xLn7nKQKPac1gMy014xsoMkOoFpASIbS8H727zIGm5coxB/x+2tuSTstIKVVJhfZkvkQSQhVySyUl0SaIuaNpKxTuE1EJjJRKQ14TXaZTqbRU/kG0EWKd+Z81zoCTz1L5TXbdJ8ivXQTKPFffoKKXbCe0LZw7Uko7DVdAD8fcM9YRTJ4i8JjWFLZttKNJhBCYJnQniu33QhMlm73oOk6Zg0i9ztJTq9F12PanDNKSSMvCsixeerqf9kPFEttt00/ER/c+pwPhiFY0S7dpmqnx4dND/P7nuUxmocGHT1d1lLY83+u06/z4XzbmZXcHqwRnfrS4hLcQgpPOqDlsH8FwzT1jWfzOUwQe0xp7pmXX83/itz3O7EzX1SwwjkldnYam4wgLTZMYRi7axM4nyGU7yWx9HpUVbJnQftDiT0/109k2dHxgKROIRz5CU8/fNhvFY6rUdn45btucU8VLz+YX6Is2amz7k4qYSqcl0lK9ozf9+iAdbf2qBEiVINKg8+rm/pJ9nYUQRfkHlTKReh14isBj2iOEQAicUsi52VlOSYD64Ra2lQzXa07imOo7rOL+7ZpA7nh1oCIlMBT1UUG8Sw7u5RxnItHKQkVHi8aZGktPrWLLc310ZvMkjEzxftEmWHpaFZYl8grm+fyQSFgYLhNSIKQcwbGOJKmkqgN11vl16Dp5fZ0Hm7kPJ8RzIvU68BSBhwflZ2d2Q3Z3/LhpCmdJb5omQgjSqVwz+kRclUgWwuLl53pHPSonHhtcA0RnCGIVFoIbLcZTCYCa/W95vt9RAuXo7VFCPNygE45odGNRF9bojpt5vgehKaEfDAqV93Gon0iDTjCohHOlcf7DDfGcKL0OJsAQPDyOPKVmZ+qHnUv2suPHdR3C9RrJATPrlFTO4UiDzrLTlAkiETfZ/Ew/HYdGJpBPOsvPS89kSjqBhQ5YxfkCsXaJz196hjwZidQX+0xMAzrbhn62trDvOGg6DXpEBOqjqg+zZarOb/VR1VlNCMF5n5zBoUMdebP0Smbuk7m/8SQZpofH2FM4OzPN/Pr7dvy4EIJjjg/SdiDneNR0WHpKdTZjWMWvd6ZGbgba+nxpJQDQEIXuLkrG5Q8nVn+iYysBf6A4kaxSnHISQHfC4iPZLm+aJouS/OyyEnaymP3OhxLqg60qJ4L5ZzA8ReDhUQa3szhSr4rFqWxVk2f/kN/ovT4qHOewjZSqlEFXiaKqPj+EGyDRlXUMF0xubaGXKTerF6BpGoZRRktMIP/BaHG4SsCNvXLL1Q+CrS/k95VQPZUtXnq2n3jMIJyNAHOXqi5F+VXlke83MBSeIvDwKEPZCqNhPc/sctbHqghHAs6sMZqtCiolaEIwa06AQ/vzg+FNQwlyy7CKhbbIL0Xh84Mmsn0MbCR0lAhB9cghRDaqyLU6OutjNdTUqql6JmM54aZ2E6DNT/fR29NHbR3E46YqAHjQ5NFf9jCjgszhkqvKSWAumoBD8vCYONg/bMPI7zrmtsG/9Uqak870OwJCE5qq+Ea2bo5I4c+Wr7DbQUoJia7SnbiQuQbyAKedW8ULT5buTTwV8fkgElUO78LnEwhCTa3KJq4Et0L1++H5P/YRadARCNoPGVhSXc8fhNpaVc7arj0Uiei0J9UJpFU6c3gos89EChEdDE8ReHhUgJ0slk4ZZFKShibo7Vb1+9sPmk6T9EK/AuD02fUF8qVabR10VZA1+/qWgTGtNTQRsR3B0RlqVZWIS+rrNZafpUwrm5/uJxE3B+2foBIDc/8ZBsgMxGMq0sv2vRgZtepyF7urrYMPnx5iy3Mpp2JpuF7LE+SVmH0mUojoYIx/TzQPjwnCcOrLCyFYeko1/oAq49DbDeFITirYTdJ1HSJRH/5A8TmMdLbZSkAVYSvlOyhFuX4CUxW3KSfWDj3dqrro8mxfAk1TTXjqwsXH+gP5M3afX/WBsKycQkinQFoSt0wujLDqbIfHf93H0tOCNM7SCYQEoiCV3G32GayPgLunQSnGqs/BcPAUgce05HAa0vh8gvqoTzWcr9dZdkaImbN1giGoj/qc2aJlmkVOXk1TCUozjvJx9sfrnOYrbppm6pz3marRuL2KqSkhTCcaqaSk/ZDB5qf6sLLTeNXMpfidZdJKkdgyN5NWQj9YJWicqePLKuh0GuqjytRUjnQKNj/dR8dB5UMoFPa22SeUzT4+HLPPaDRGGg3GxDR01113sX//fgD6+/uprq7mtttuK9pv1apVhEIhNE1D13U2btw4FsPx8CjicJx4TlvBZ5RZYsvzSZafqRrS2Mv+dNrMzx0QUFWlURsRfPjUmqzzUiKt4h98d7cJqI5m4yUP+rqH3udI4g4ZbW+z2PxMPyedWePkcnQcshBCVRAVQtB+SElq9zNMZyCgqUzsunBuhdXVmVMEwSrBGStqMDIWT/4uFxHmzl8Ih1VpEZWNMDpmn4niTB6TS15xxRXOv//rv/6L6urqsvvedNNNhMOTYFriMaWoxIlXqumIZQm649kZYrastLuEcaF5wO8HEAiUsrBDEtMlQiHTSdj6p96iSJdpTYFs7YoZDAyYSMtyoqakhGVnVKHruqOkwxEN0zDpTiizTzoJ7UmTxhkCd5iWlNnSEhFNJZRV6cxdEKLtUJJIvYamCeIx1REtkSjuKTzSzOCJ4kweU90jpeT555/nxhtvHMvLeHgMm6Fmc+6mIxJVvfLks+ryq5XWa3nVSgGCQeHMYjVdhYlm0qr+UColc83ss/ZqpJrNxrssjAzE2txjHGJlkC/TJj6HMd5MoTPdgj883Fu030vPJDnlnFqWn6EawXfFVPSP7TC2n2N3t8xbZUigvt5HImHy0rP9LDtdlZnWNIGmayzPNi166tGePPPQaM3aJ4ozeUx9BG+99RaRSISjjjqq7D4bNmzg2muvZdOmTWM5FA+PIgZz4pkmTjljaamYfcOQzg/3nGxm6hO/7XFsu3YoYUOjTjAkaJyhY1uAJOqHXhcRzgfRJsH5F9Rx8tm1RBuLp4JNs3TO/XQNTbM0RPaXKjRVcG3FZ6tpmqnhD9irjolPuVaQNoWvQQg1Ww8EVXhn00ytrMTq7lYC2m4En0m7soklNM0QBEMQbfSx4jO1zJilEwhBNKqTiBukBqSzUutsT5EakHR3qRVfMCioH4EvYChn8FDO5PHgsPXaunXriMfjRZ9//vOfZ/ny5QA8++yznH766YOeIxqNkkgkWL9+PXPmzOHYY48tue+mTZscZbFx40aamprKntfn8w26fTIzVe9tot2XlJKmGRn27VH2YiFUv21N05wM4p7EbpIDEk2TRCJRnnqsjY62AdIpiWlK+nogEFC5A4GAxqxZTQQCBqByAmIdkqamJnw+wTkfq+fpTW0c2JeLE9V9fp5/fIBoU4CZsyVdnSmiTUF8Po1nNyUZ6B+7hDJ37+XhEAwJNE1F6aQKGu/4/QJLl2UzhKWEGTN9tLcpu1ioSuczfzsXTYenHmsnHkujifxlhc+vzHaNM6poaGjA79eYMcug/VA/6ZTqEhcMaXzirxcgpeYI3E/9jRLOmib56Y/3IISJEDpHHdVI0yyTjkNJGmeEmDWrCSEEn7igCcOQwxbYUkr+8JuDdLar8533ydkTMoRUyDFyU5umycUXX8zGjRtpbGwccv+f/vSnhEIhPvvZz1Z0ftsZXYqmpiY6OiqMzZtkTNV7G+/7qqT+i2panq082uADpFOTftnp1cre36XKT5x4ShVP/q43qxjAHxRE6nXiXQbppHJGnnV+nWNiAKVcVny2hid/10c6pWa90lI2bbc5o7DPbql2lEIAonz/YSEgOkOjsy23gy9AXhnm0SBYJTj93FoCQYvHH+nL84UEQtmkrUFes+7H6SdQqodzsEpw6jnVPPOHPoy0+jtcJ+hoV07jplkqnt+yBGDy0rNJuhPKjHf88mqCQeF0GAPluH/8N91OA5pzPxlm1qymoqJzh4thSP74m26SA5JQleAjnwyXbIs5XsyZM6fk52NmGnrttdeYM2dOWSWQTCYZGBhw/v3qq6+yYMGCsRqOh4dDpSF7mqZx8lm1nPepCEtPrXb6FSSyJoNlp1cTiWjEuwy2vNBPOGs+aJql85FP1LHs9CoiEY1gCMJhgd8vs01tlJCLzhRsfX7AmXlnUsqvoMaYG0eh2Cg1XCmLlYBbhjU0wYmnhHInExCJVPa8ItHBBZfPr+5H01So5rN/6GHrCynO+0wtTTNdx1qDKwGA+nqNQDCrCK3i+kKRep3qGg07+lZakkS3ytBW/Y3dDnydnoRy7LcfMNn0qx4e+1W3E4IKuXpSoSpl/skVmBsdU81ohJiOB2PmLC5lForFYnz/+99n9erVJBIJbr/9dkCtHs444wxOOOGEsRqOh4fDcEL27KgQKYudxIYh6WhTfoTOQyYf/cu6bCi0OvbFp/tob1O1hDrTFpuf7WfpKbkIuq0vDNB2IBceFAjklzUAlVug6aqpih1yGo6oWkZDUd+YS1qLtaPCIu3DJHRV2EMgMUT/g9NXhKiqCmAYkqcf7SGVVM8XdE45J0wmY/Hyc30layMVOsQ1XXDOJ2rZ9mKSRJe6Z1sZqNl+NZYl8pRcpF6n/WCuFIQd4mkLYUsaTmZ2OqWc9lVV9vWVQk+n1fMfbbPNRHEGD8WYmYbGGs80NLUYz/uqtCJkofnIFv5bX8gWn4toKm5dqtnw+ReE8fvVItswJI//OqEihLIITWW51jfonHhKNX/8bc5M5PPDRz9bi6ZpbH5aNU1XzdLV2GxnoxBKwP3pqd58wVoiImdYdn6Rbe5+GK9gxiydk89WDeA3P9OnKnZGNE48tdoZ++OP9DoCv3GmzvIzQqRTGs/8oaeofMbM2bqTn+EuE52rGJr//k44uYo//qaHdEol7Z37qUheGWnLsnjit73Os5gxW0WAqXBgywk5rc9+F2bMmDElf2NQ3jTk1RrymHZUMkuzhU08ZhCpV4JJ0zSEyK0m0imlBISAxll6nu3Xno22HzRzNW/AyT8ANZNtG1ArAmW21tE04dQscisgW/mEG3SWnlLNSWfWsPmZrI8iIjhueRWP/zq/NHZdpHThNjc+H2g+NZPu7VXXLFcqoRzd3ZbT2vPDp1Xx4tP9tLeZPPqL3uzzzp/1dydMtr6Q4sOnVREJa7QXOJUTcdNJ0jNNtSIrbBrvfn8ADY0+RzHoerGyP+cTtY4y6EnYCl6y+em+XOMhypeJmOp4isBjWjJUIpBpqo5kqSS0HTTzMlojDToSk0y2kmggCB8+taZIWJ10Zi2GIYlGo3R2drL1TwN0ZwWT6oesYt7tNpi2UBuslHE6ZfDH33Rn22ZWO93QXnspme9gzvYraJpN1sRSup6/BE49u5bnn+gtW9hOaCrXIdGlwjKFAN0HPp+GJS0i2bac6TS8/EwvHQVtMgsVUSYFXZ2GGnu3pbJ7bUe3gHC9jmEYvPxcikRcPa+lp1Tn2e0Ln5G7XLjdK9pt/tM0rUhZlGs8NB3xFIGHRwl0PTtjz84WE3HTySNYdno1pglbnu9zoohKRYIIIfD7BcGgj0BA56SCVYgQxbP/smNxKR/bBp/J5LKcu6WkcaaWl23bnbA486O1+HzKxPJSdgXhjrHXdXjhqT5ARS0VJXABTTN0Tjy1iid+p2b4gZDgzI/WMmdOE7/75Xsk4iabHu7JO2857AioTDrXyS0YgrM+Fsbns9j8dB/tB00e+2Wfc0yHS/ktO706r6SH+1nrumotmuhSfpywK2NXKd7856/rkvqoL7fqO6N6wtrwxxpPEXh4lEAIwfKs+SURN4nUa455xvYrnHTm0ELcply4aqUlCk48pdoxEdnKJxAocGAXXtNSzlu1eqjhpLPUCmWLbWYKa3R3W07o5Jkranjq971FK4d43GTLCwNOqGk6KXl1cz+zPo2jiArRfXDWx6p55rH+vPPZkU22uSyYjdYJBGDzM8mSTnAJpJJqhbb5mX6646V9O/lBAJaT9Od+5u5nPVkcueOBpwg8PMqgaRonnVnjmBqe+G1PQaRRZUJ8JO0KSx3rnhHbgkxKyR9/2+Mc1zRD0N2TWz3Y4/X7hbMycbquZWfNoZDGuZ+qI5OxeOXFfmK2iUeq5u+5MUF3XEl0e6ViR/c4Kw0fBAI6DVG9aBVil+KO1GssPa0Gv18r2cdBCBU1hZD0JCThiKZ6EBREe9lKVtNkXt0et3PZ3Xs4/xojqxU0VfAegYfHIJQKHx1uPLhhyMOuMFkY6lpY5M49vsL+yi8/N5BnEy88Bkq34ow06Jxydh2mqcb+1KM9hcOirl7D79fyjjcMyZbn+uhOWESiPrWCSliEIzqaDh3ZyqDRmRqaUEJ9258GWH5G1vdSr9GeMhGoyCK7T7D9HAoVVymncKHpaLL0DD7SeIrAw6MCRmJG8PnEYSuRSqtTloqkOTGbszBYclTJVpzSwDTB79fw+SQNUR+xTiPPf3D88mrHz2ErlUAATs4qEHuFkhqQpAZMZszSWPHZWkdIP/m7XieCyl6tLD21xjmmO2Flo7TcJp3id1CoZEtVg50IZZ4nOt4j8fCokMM1I4xEiQzn2NzqoHgWPBR2ff90Splxtjzfz9JTqx0nayZjObH6gaCqsjr4GPLDYxNxi21/StIdL3bk2opL13GS5qQl82r/u30spcJ0yynKiVLmeaLjKQIPj3FgJLboSo+1hWVh6GSlTXfcM/KOQ+4w1RoCAZ2PfjbsZOBqQ5QSVQosFx4bzpbiUCGqpR257oxhIchWE5VFZqvCfgCDKUrPIVwZniLw8JgCuFcB4QadcL0GWMOaBft8wvEzuMNUbUWiaRqhUOVj0jSNk86qJZOx2PJCP0ZaJc6F67WS5ipdVy0/7fBPJ4muXifRZWSztIsV21CKcrDtlRQfnA54isDDYwpQaAsvNeMeCnv2bBgyL0dipOaUrc/3O7WAglVq5VFqTIVRUHaUlsw6wLsTozMeG8+RnMNTBB4eU4BCW/jhVs+0k+CGkyMxGKWydwcrw1wuSqtUItlIZ/OeIznHNL1tD4+pxWjbwkcrvt429ww3e7fU/bjdEqMxm/ccyTk8ReDhMUWYiMlRw1VQhbP8cvczGrN5z5GcY4J9bTw8PCYCo+lEHU7UU6Wz/NGazU9E5Xkk8B6Bh4dHHkfKiTrchkHebH70GJEieP755/m///f/sm/fPm6++WYWL17sbPv5z3/O448/jqZpfOUrXynZfaytrY1/+7d/o6enh5aWFi677DJ8nnr28DiiHCkn6nBn+d5sfvQYUc/i+fPnc/XVV3PMMcfkfb53716ee+457rzzTm644QYeeOCBvD6hNg899BCf+tSnuPvuu6mpqeHxxx8fyXA8PDxGgSPVZ9ee5X/kk+FpHcp5JBiRIpg3b17J1mebN2/mtNNOw+/3M3PmTGbPns327dvz9pFS8sYbb3DKKacAcM4557B58+aRDMfDw2MUOJICeTQbx3tUzpgsrGKxGEcffbTzdzQaJRbL75Td09NDdXU1ena6UWofN5s2bWLTpk0AbNy4kaamprL7+ny+QbdPZqbqvU3V+4Kpe2/efU0dhlQE69atIx6PF33++c9/nuXLl4/FmEqyYsUKVqxY4fw9WHPpqdrgHabuvU3V+4KJd28jiQhyHztVm7xPtPc1mhx28/q1a9cO+2J2j1abWCxGNBrN26euro7+/n5M00TX9ZL7eHh4jC6j2STnExdMr1nzVGZEPoJyLFu2jOeee45MJkNbWxsHDhxgyZIlefsIIfjgBz/ICy+8AMATTzzBsmXLxmI4Hh4eWdwRQXZBucM91jCGaFDsMWkYkSJ48cUXufjii3nnnXfYuHEjGzZsAFQ00amnnsqVV17Jhg0b+Kd/+ienbO0tt9zi+AL+4R/+gUceeYTLLruM3t5ezj333BHejoeHx2CMJCKo8NjBagZ5TC6ElHJSqvX9+/eX3TaVbXxT9d6m6n3BxLs3z0cwOBPtfY0mh+0j8PDwmFqMR5Mcj8nFmPgIPDw8PDwmD54i8PDw8JjmeIrAw8PDY5rjKQIPDw+PaY6nCDw8PDymOZ4i8PDw8JjmTNo8Ag8PDw+P0WFKrgiuu+66Iz2EMWOq3ttUvS+Yuvfm3dfUYUoqAg8PDw+PyvEUgYeHh8c0Z0oqAnffgqnGVL23qXpfMHXvzbuvqYPnLPbw8PCY5kzJFYGHh4eHR+V4isDDw8NjmjNlC8ru2rWLH/zgB6TTaXRd52tf+1pRl7TJym9/+1t+//vfo2kaS5cu5R//8R+P9JBGlYcffpif/OQn/PCHPyQcDh/p4YyYn/zkJ7z88sv4fD5mzZrFypUrqampOdLDOmy2bdvGgw8+iGVZnHfeeVxwwQVHekijQkdHB/feey/xeBwhBCtWrOCTn/zkkR7WuDBlFcFDDz3E5z73OU488US2bNnCQw89xL/+678e6WGNmNdff52XXnqJ2267Db/fTyKRONJDGlU6Ojp49dVXaWqaOv1wjzvuOP7+7/8eXdd56KGH+PnPfz5plbdlWTzwwAOsWbOGxsZGVq9ezbJly5g3b96RHtqI0XWdCy+8kJaWFgYGBrjuuus47rjjpsS9DcWUNQ0JIRgYGACgv7+fhoaGIzyi0eHRRx/lL//yL/H7/QBEIpEjPKLR5cc//jH/8A//MOzOWROZ448/Hj3bE/J973uf06p1MrJ9+3Zmz57NrFmz8Pl8nHbaaWzevPlID2tUaGhooKWlBYCqqirmzp07qd/VcJiyK4IvfelLbNiwgZ/85CdYlsX69euP9JBGhQMHDvD222/zf/7P/8Hv93PhhRdOGZPX5s2biUajLFq06EgPZcx4/PHHOe200470MA6bWCxGY2Oj83djYyPvvvvuERzR2NDW1kZra+uU+W0NxaRWBOvWrSMejxd9/vnPf57XXnuNL33pS5xyyik899xz3H///axdu3b8B3kYDHZflmXR29vLhg0b2LFjB3fddRf33HPPpJlBD3ZvP//5z1mzZs34D2oUGOy+li9fDsDPfvYzdF3nzDPPHOfReQyHZDLJHXfcwZe//GWqq6uP9HDGhUmtCAYT7Pfccw9f+cpXADj11FP5/ve/P17DGjGD3dejjz7KSSedhBCCJUuWoGkaPT09k8apWu7e9uzZQ1tbG9dccw0AnZ2dXHvttdxyyy3U19eP4wgPj6EmGU888QQvv/wyN95446RR2qWIRqN0dnY6f3d2dhKNRo/giEYXwzC44447OPPMMzn55JOP9HDGjSnrI4hGo7z55puAcrDOnj37CI9odFi+fDlvvPEGAPv378cwDOrq6o7wqEbOggUL+OEPf8i9997LvffeS2NjI7feeuukUAJDsW3bNn75y19y7bXXEgwGj/RwRsTixYs5cOAAbW1tGIbBc889x7Jly470sEYFKSX3338/c+fO5dOf/vSRHs64MmUzi99++20nxM3v9/O1r33NcQRNZgzD4L777mP37t34fD4uvPBC/uIv/uJID2vUWbVqFbfccsukWekMxmWXXYZhGNTW1gJw9NFHc9FFFx3hUR0+W7Zs4cc//jGWZfGRj3yEv/7rvz7SQxoV3n77bW688UYWLFjgrNq+8IUvsHTp0iM8srFnyioCDw8PD4/KmLKmIQ8PDw+PyvAUgYeHh8c0x1MEHh4eHtMcTxF4eHh4THM8ReDh4eExzfEUgYeHh8c0x1MEHh4eHtOc/w9TU12YN0HIgAAAAABJRU5ErkJggg==\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 生成符合正态分布的聚类数据\n",
    "from sklearn import datasets\n",
    "x, y = datasets.make_blobs(n_samples=5000, n_features=2, centers=3)\n",
    "for _,c in enumerate(np.unique(y)):\n",
    "    plt.scatter(x[y==c, 0], x[y==c, 1],s=7)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**在本开源项目的章节中，我们要重点介绍回归和分类的集成学习的问题，因此我们在接下来的章节中不再介绍关于无监督学习的具体算法，后面的内容仅仅涉及回归和分类问题。**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. 使用sklearn构建完整的机器学习项目流程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一般来说，一个完整的机器学习项目分为以下步骤：\n",
    "   - 明确项目任务：回归/分类\n",
    "   - 收集数据集并选择合适的特征。\n",
    "   - 选择度量模型性能的指标。\n",
    "   - 选择具体的模型并进行训练以优化模型。\n",
    "   - 评估模型的性能并调参。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 使用sklearn构建完整的回归项目"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(1) 收集数据集并选择合适的特征：        \n",
    "在数据集上我们使用我们比较熟悉的Boston房价数据集，原因是：        \n",
    "   - 第一个，我们通过这些简单的数据集快速让我们上手sklearn，以及掌握sklearn的相关操作。\n",
    "   - 第二个，我们用简单的数据集能更加清晰地介绍机器学习的相关模型，避免在处理数据上花费较大的精力。\n",
    "\n",
    "如果您对具体的项目感兴趣，我们会在第六章给出三个大型的案例让大家体验。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:47:24.395506Z",
     "start_time": "2021-03-14T11:47:24.372506Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "boston = datasets.load_boston()     # 返回一个类似于字典的类\n",
    "X = boston.data\n",
    "y = boston.target\n",
    "features = boston.feature_names\n",
    "boston_data = pd.DataFrame(X,columns=features)\n",
    "boston_data[\"Price\"] = y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:47:26.933502Z",
     "start_time": "2021-03-14T11:47:26.908501Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "        CRIM    ZN  INDUS  CHAS    NOX     RM   AGE     DIS  RAD    TAX  \\\n0    0.00632  18.0   2.31   0.0  0.538  6.575  65.2  4.0900  1.0  296.0   \n1    0.02731   0.0   7.07   0.0  0.469  6.421  78.9  4.9671  2.0  242.0   \n2    0.02729   0.0   7.07   0.0  0.469  7.185  61.1  4.9671  2.0  242.0   \n3    0.03237   0.0   2.18   0.0  0.458  6.998  45.8  6.0622  3.0  222.0   \n4    0.06905   0.0   2.18   0.0  0.458  7.147  54.2  6.0622  3.0  222.0   \n..       ...   ...    ...   ...    ...    ...   ...     ...  ...    ...   \n501  0.06263   0.0  11.93   0.0  0.573  6.593  69.1  2.4786  1.0  273.0   \n502  0.04527   0.0  11.93   0.0  0.573  6.120  76.7  2.2875  1.0  273.0   \n503  0.06076   0.0  11.93   0.0  0.573  6.976  91.0  2.1675  1.0  273.0   \n504  0.10959   0.0  11.93   0.0  0.573  6.794  89.3  2.3889  1.0  273.0   \n505  0.04741   0.0  11.93   0.0  0.573  6.030  80.8  2.5050  1.0  273.0   \n\n     PTRATIO       B  LSTAT  Price  \n0       15.3  396.90   4.98   24.0  \n1       17.8  396.90   9.14   21.6  \n2       17.8  392.83   4.03   34.7  \n3       18.7  394.63   2.94   33.4  \n4       18.7  396.90   5.33   36.2  \n..       ...     ...    ...    ...  \n501     21.0  391.99   9.67   22.4  \n502     21.0  396.90   9.08   20.6  \n503     21.0  396.90   5.64   23.9  \n504     21.0  393.45   6.48   22.0  \n505     21.0  396.90   7.88   11.9  \n\n[506 rows x 14 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>CRIM</th>\n      <th>ZN</th>\n      <th>INDUS</th>\n      <th>CHAS</th>\n      <th>NOX</th>\n      <th>RM</th>\n      <th>AGE</th>\n      <th>DIS</th>\n      <th>RAD</th>\n      <th>TAX</th>\n      <th>PTRATIO</th>\n      <th>B</th>\n      <th>LSTAT</th>\n      <th>Price</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.00632</td>\n      <td>18.0</td>\n      <td>2.31</td>\n      <td>0.0</td>\n      <td>0.538</td>\n      <td>6.575</td>\n      <td>65.2</td>\n      <td>4.0900</td>\n      <td>1.0</td>\n      <td>296.0</td>\n      <td>15.3</td>\n      <td>396.90</td>\n      <td>4.98</td>\n      <td>24.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.02731</td>\n      <td>0.0</td>\n      <td>7.07</td>\n      <td>0.0</td>\n      <td>0.469</td>\n      <td>6.421</td>\n      <td>78.9</td>\n      <td>4.9671</td>\n      <td>2.0</td>\n      <td>242.0</td>\n      <td>17.8</td>\n      <td>396.90</td>\n      <td>9.14</td>\n      <td>21.6</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.02729</td>\n      <td>0.0</td>\n      <td>7.07</td>\n      <td>0.0</td>\n      <td>0.469</td>\n      <td>7.185</td>\n      <td>61.1</td>\n      <td>4.9671</td>\n      <td>2.0</td>\n      <td>242.0</td>\n      <td>17.8</td>\n      <td>392.83</td>\n      <td>4.03</td>\n      <td>34.7</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.03237</td>\n      <td>0.0</td>\n      <td>2.18</td>\n      <td>0.0</td>\n      <td>0.458</td>\n      <td>6.998</td>\n      <td>45.8</td>\n      <td>6.0622</td>\n      <td>3.0</td>\n      <td>222.0</td>\n      <td>18.7</td>\n      <td>394.63</td>\n      <td>2.94</td>\n      <td>33.4</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.06905</td>\n      <td>0.0</td>\n      <td>2.18</td>\n      <td>0.0</td>\n      <td>0.458</td>\n      <td>7.147</td>\n      <td>54.2</td>\n      <td>6.0622</td>\n      <td>3.0</td>\n      <td>222.0</td>\n      <td>18.7</td>\n      <td>396.90</td>\n      <td>5.33</td>\n      <td>36.2</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>501</th>\n      <td>0.06263</td>\n      <td>0.0</td>\n      <td>11.93</td>\n      <td>0.0</td>\n      <td>0.573</td>\n      <td>6.593</td>\n      <td>69.1</td>\n      <td>2.4786</td>\n      <td>1.0</td>\n      <td>273.0</td>\n      <td>21.0</td>\n      <td>391.99</td>\n      <td>9.67</td>\n      <td>22.4</td>\n    </tr>\n    <tr>\n      <th>502</th>\n      <td>0.04527</td>\n      <td>0.0</td>\n      <td>11.93</td>\n      <td>0.0</td>\n      <td>0.573</td>\n      <td>6.120</td>\n      <td>76.7</td>\n      <td>2.2875</td>\n      <td>1.0</td>\n      <td>273.0</td>\n      <td>21.0</td>\n      <td>396.90</td>\n      <td>9.08</td>\n      <td>20.6</td>\n    </tr>\n    <tr>\n      <th>503</th>\n      <td>0.06076</td>\n      <td>0.0</td>\n      <td>11.93</td>\n      <td>0.0</td>\n      <td>0.573</td>\n      <td>6.976</td>\n      <td>91.0</td>\n      <td>2.1675</td>\n      <td>1.0</td>\n      <td>273.0</td>\n      <td>21.0</td>\n      <td>396.90</td>\n      <td>5.64</td>\n      <td>23.9</td>\n    </tr>\n    <tr>\n      <th>504</th>\n      <td>0.10959</td>\n      <td>0.0</td>\n      <td>11.93</td>\n      <td>0.0</td>\n      <td>0.573</td>\n      <td>6.794</td>\n      <td>89.3</td>\n      <td>2.3889</td>\n      <td>1.0</td>\n      <td>273.0</td>\n      <td>21.0</td>\n      <td>393.45</td>\n      <td>6.48</td>\n      <td>22.0</td>\n    </tr>\n    <tr>\n      <th>505</th>\n      <td>0.04741</td>\n      <td>0.0</td>\n      <td>11.93</td>\n      <td>0.0</td>\n      <td>0.573</td>\n      <td>6.030</td>\n      <td>80.8</td>\n      <td>2.5050</td>\n      <td>1.0</td>\n      <td>273.0</td>\n      <td>21.0</td>\n      <td>396.90</td>\n      <td>7.88</td>\n      <td>11.9</td>\n    </tr>\n  </tbody>\n</table>\n<p>506 rows × 14 columns</p>\n</div>"
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boston_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "各个特征的相关解释：\n",
    "   - CRIM：各城镇的人均犯罪率\n",
    "   - ZN：规划地段超过25,000平方英尺的住宅用地比例\n",
    "   - INDUS：城镇非零售商业用地比例\n",
    "   - CHAS：是否在查尔斯河边(=1是)\n",
    "   - NOX：一氧化氮浓度(/千万分之一)\n",
    "   - RM：每个住宅的平均房间数\n",
    "   - AGE：1940年以前建造的自住房屋的比例\n",
    "   - DIS：到波士顿五个就业中心的加权距离\n",
    "   - RAD：放射状公路的可达性指数\n",
    "   - TAX：全部价值的房产税率(每1万美元)\n",
    "   - PTRATIO：按城镇分配的学生与教师比例\n",
    "   - B：1000(Bk - 0.63)^2其中Bk是每个城镇的黑人比例\n",
    "   - LSTAT：较低地位人口\n",
    "   - Price：房价"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(2) 选择度量模型性能的指标：                      \n",
    "   - MSE均方误差：$\\text{MSE}(y, \\hat{y}) = \\frac{1}{n_\\text{samples}} \\sum_{i=0}^{n_\\text{samples} - 1} (y_i - \\hat{y}_i)^2.$\n",
    "   - MAE平均绝对误差:$\\text{MAE}(y, \\hat{y}) = \\frac{1}{n_{\\text{samples}}} \\sum_{i=0}^{n_{\\text{samples}}-1} \\left| y_i - \\hat{y}_i \\right|$\n",
    "   - $R^2$决定系数：$R^2(y, \\hat{y}) = 1 - \\frac{\\sum_{i=1}^{n} (y_i - \\hat{y}_i)^2}{\\sum_{i=1}^{n} (y_i - \\bar{y})^2}$\n",
    "   - 解释方差得分:$explained\\_{}variance(y, \\hat{y}) = 1 - \\frac{Var\\{ y - \\hat{y}\\}}{Var\\{y\\}}$\n",
    "\n",
    "https://scikit-learn.org/stable/modules/model_evaluation.html#regression-metrics\n",
    "\n",
    "![jupyter](./1.3.png)              \n",
    "在这个案例中，我们使用MSE均方误差为模型的性能度量指标。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(3) 选择具体的模型并进行训练\n",
    "   - **线性回归模型**         \n",
    "   回归这个概念是19世纪80年代由英国统计学家郎西斯.高尔顿在研究父子身高关系提出来的，他发现：在同一族群中，子代的平均身高介于父代的身高以及族群的平均身高之间。具体而言，高个子父亲的儿子的身高有低于其父亲身高的趋势，而矮个子父亲的儿子身高则有高于父亲的身高的趋势。也就是说，子代的身高有向族群平均身高\"平均\"的趋势，这就是统计学上\"回归\"的最初含义。回归分析是一种预测性的建模技术，它研究的是因变量（目标）和自变量（特征）之间的关系。这种技术通常用于预测分析，时间序列模型以及发现变量之间的因果关系。通常使用曲线/线来拟合数据点，目标是使曲线到数据点的距离差异最小。而线性回归就是回归问题中的一种，线性回归假设目标值与特征之间线性相关，即满足一个多元一次方程。通过构建损失函数，来求解损失函数最小时的参数w ：                                                \n",
    "   假设：数据集$D = \\{(x_1,y_1),...,(x_N,y_N) \\}$，$x_i \\in R^p,y_i \\in R,i = 1,2,...,N$，$X = (x_1,x_2,...,x_N)^T,Y=(y_1,y_2,...,y_N)^T$                        \n",
    "   假设X和Y之间存在线性关系，模型的具体形式为$\\hat{y}=f(w) =w^Tx$           \n",
    "   ![jupyter](./1.4.png)      \n",
    "   (a) 最小二乘估计：                 \n",
    "   我们需要衡量真实值$y_i$与线性回归模型的预测值$w^Tx_i$之间的差距，在这里我们和使用二范数的平方和L(w)来描述这种差距，即：                      \n",
    "   $$\n",
    "   L(w) = \\sum\\limits_{i=1}^{N}||w^Tx_i-y_i||_2^2=\\sum\\limits_{i=1}^{N}(w^Tx_i-y_i)^2 = (w^TX^T-Y^T)(w^TX^T-Y^T)^T = w^TX^TXw - 2w^TX^TY+YY^T\\\\\n",
    "   因此，我们需要找到使得L(w)最小时对应的参数w，即：\\\\\n",
    "   \\hat{w} = argmin\\;L(w)\\\\\n",
    "   为了达到求解最小化L(w)问题，我们应用高等数学的知识，使用求导来解决这个问题： \\\\\n",
    "   \\frac{\\partial L(w)}{\\partial w} = 2X^TXw-2X^TY = 0,因此： \\\\\n",
    "   \\hat{w} = (X^TX)^{-1}X^TY\n",
    "   $$                   \n",
    "   (b) 几何解释：                \n",
    "   在线性代数中，我们知道两个向量a和b相互垂直可以得出：$<a,b> = a.b = a^Tb = 0$,而平面X的法向量为Y-Xw，与平面X互相垂直，因此：$X^T(Y-Xw) = 0$，即：$w = (X^TX)^{-1}X^TY$                             \n",
    "   ![jupyter](./1.5.png)               \n",
    "   (c) 概率视角：       \n",
    "   假设噪声$\\epsilon \\backsim N(0,\\sigma^2),y=f(w)+\\epsilon=w^Tx+\\epsilon$，因此：$y|x_i,w ~ N(w^Tx,\\sigma^2)$          \n",
    "   我们使用极大似然估计MLE对参数w进行估计：       \n",
    "   $$\n",
    "   L(w) = log\\;P(Y|X;w) = log\\;\\prod_{i=1}^N P(y_i|x_i;w) = \\sum\\limits_{i=1}^{N} log\\; P(y_i|x_i;w)\\\\\n",
    "    = \\sum\\limits_{i=1}^{N}log(\\frac{1}{\\sqrt{2\\pi \\sigma}}exp(-\\frac{(y_i-w^Tx_i)^2}{2\\sigma^2})) = \\sum\\limits_{i=1}^{N}[log(\\frac{1}{\\sqrt{2\\pi}\\sigma})-\\frac{1}{2\\sigma^2}(y_i-w^Tx_i)^2] \\\\\n",
    "    argmax_w L(w) = argmin_w[l(w) = \\sum\\limits_{i = 1}^{N}(y_i-w^Tx_i)^2]\\\\\n",
    "    因此：线性回归的最小二乘估计<==>噪声\\epsilon\\backsim N(0,\\sigma^2)的极大似然估计\n",
    "   $$\n",
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面，我们使用sklearn的线性回归实例来演示：                   \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html#sklearn.linear_model.LinearRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:47:46.070536Z",
     "start_time": "2021-03-14T11:47:46.055536Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "模型系数： [-1.08011358e-01  4.64204584e-02  2.05586264e-02  2.68673382e+00\n",
      " -1.77666112e+01  3.80986521e+00  6.92224640e-04 -1.47556685e+00\n",
      "  3.06049479e-01 -1.23345939e-02 -9.52747232e-01  9.31168327e-03\n",
      " -5.24758378e-01]\n",
      "模型得分： 0.7406426641094095\n"
     ]
    }
   ],
   "source": [
    "from sklearn import linear_model      # 引入线性回归方法\n",
    "lin_reg = linear_model.LinearRegression()       # 创建线性回归的类\n",
    "lin_reg.fit(X,y)        # 输入特征X和因变量y进行训练\n",
    "print(\"模型系数：\",lin_reg.coef_)             # 输出模型的系数\n",
    "print(\"模型得分：\",lin_reg.score(X,y))    # 输出模型的决定系数R^2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - **线性回归的推广**                     \n",
    "   在线性回归中，我们假设因变量与特征之间的关系是线性关系，这样的假设使得模型很简单，但是缺点也是显然的，那就是当数据存在非线性关系时，我们使用线性回归模型进行预测会导致预测性能极其低下，因为模型的形式本身是线性的，无法表达数据中的非线性关系。我们一个很自然的想法就是去推广线性回归模型，使得推广后的模型更能表达非线性的关系。                     \n",
    "   (a) 多项式回归：        \n",
    "   为了体现因变量和特征的非线性关系，一个很自然而然的想法就是将标准的线性回归模型：             \n",
    "   $$\n",
    "   y_i = w_0 + w_1x_i + \\epsilon_i\n",
    "   $$               \n",
    "   换成一个多项式函数：          \n",
    "   $$\n",
    "   y_i = w_0 + w_1x_i + w_2x_i^2 + ...+w_dx_i^d + \\epsilon\n",
    "   $$                        \n",
    "   对于多项式的阶数d不能取过大，一般不大于3或者4，因为d越大，多项式曲线就会越光滑，在X的边界处有异常的波动。（图中的边界处的4阶多项式拟合曲线的置信区间(虚线表示置信区间)明显增大，预测效果的稳定性下降。）   \n",
    "\n",
    "   ![jupyter](./1.6.1.png) ![jupyter](./1.6.2.png)                             \n",
    "   \n",
    "   (b) 广义可加模型(GAM)：         \n",
    "   广义可加模型GAM实际上是线性模型推广至非线性模型的一个框架，在这个框架中，每一个变量都用一个非线性函数来代替，但是模型本身保持整体可加性。GAM模型不仅仅可以用在线性回归的推广，还可以将线性分类模型进行推广。具体的推广形式是：          \n",
    "   标准的线性回归模型：           \n",
    "   $$\n",
    "   y_i = w_0 + w_1x_{i1} +...+w_px_{ip} + \\epsilon_i \n",
    "   $$                 \n",
    "   GAM模型框架：                  \n",
    "   $$\n",
    "   y_i = w_0 + \\sum\\limits_{j=1}^{p}f_{j}(x_{ij}) + \\epsilon_i\n",
    "   $$                  \n",
    "   GAM模型的优点与不足：             \n",
    "      - 优点：简单容易操作，能够很自然地推广线性回归模型至非线性模型，使得模型的预测精度有所上升；由于模型本身是可加的，因此GAM还是能像线性回归模型一样把其他因素控制不变的情况下单独对某个变量进行推断，极大地保留了线性回归的易于推断的性质。\n",
    "      - 缺点：GAM模型会经常忽略一些有意义的交互作用，比如某两个特征共同影响因变量，不过GAM还是能像线性回归一样加入交互项$x^{(i)} \\times x^{(j)}$的形式进行建模；但是GAM模型本质上还是一个可加模型，如果我们能摆脱可加性模型形式，可能还会提升模型预测精度，详情请看后面的算法。\n",
    "     \n",
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(a) 多项式回归实例介绍：                    \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html?highlight=poly#sklearn.preprocessing.PolynomialFeatures                              \n",
    "sklearn.preprocessing.PolynomialFeatures(degree=2, *, interaction_only=False, include_bias=True, order='C'):               \n",
    "   - 参数：         \n",
    "   degree：特征转换的阶数。                       \n",
    "   interaction_onlyboolean：是否只包含交互项，默认False 。              \n",
    "   include_bias：是否包含截距项，默认True。           \n",
    "   order：str in {‘C’, ‘F’}, default ‘C’，输出数组的顺序。     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "原始X为：\n",
      " [[0 1]\n",
      " [2 3]\n",
      " [4 5]]\n",
      "2次转化X：\n",
      " [[ 1.  0.  1.  0.  0.  1.]\n",
      " [ 1.  2.  3.  4.  6.  9.]\n",
      " [ 1.  4.  5. 16. 20. 25.]]\n",
      "2次转化X：\n",
      " [[ 1.  0.  1.  0.]\n",
      " [ 1.  2.  3.  6.]\n",
      " [ 1.  4.  5. 20.]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "X_arr = np.arange(6).reshape(3, 2)\n",
    "print(\"原始X为：\\n\",X_arr)\n",
    "\n",
    "poly = PolynomialFeatures(2)\n",
    "print(\"2次转化X：\\n\",poly.fit_transform(X_arr)) #[x_0,x_1,x_2,x_1x_2,x_1^2,x_2^2] \n",
    "\n",
    "poly = PolynomialFeatures(interaction_only=True)\n",
    "print(\"2次转化X：\\n\",poly.fit_transform(X_arr))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(b) GAM模型实例介绍：          \n",
    "安装pygam：pip install pygam               \n",
    "https://github.com/dswah/pyGAM/blob/master/doc/source/notebooks/quick_start.ipynb                     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:53:47.825598Z",
     "start_time": "2021-03-14T11:53:47.286600Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                    103.2423\n",
      "Link Function:                     IdentityLink Log Likelihood:                                 -1589.7653\n",
      "Number of Samples:                          506 AIC:                                             3388.0152\n",
      "                                                AICc:                                            3442.7649\n",
      "                                                GCV:                                               13.7683\n",
      "                                                Scale:                                              8.8269\n",
      "                                                Pseudo R-Squared:                                   0.9168\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                20           11.2         2.20e-11     ***         \n",
      "s(1)                              [0.6]                20           12.7         8.15e-02     .           \n",
      "s(2)                              [0.6]                20           13.3         2.59e-03     **          \n",
      "s(3)                              [0.6]                20           3.5          2.76e-01                 \n",
      "s(4)                              [0.6]                20           11.4         1.11e-16     ***         \n",
      "s(5)                              [0.6]                20           10.2         1.11e-16     ***         \n",
      "s(6)                              [0.6]                20           10.4         8.22e-01                 \n",
      "s(7)                              [0.6]                20           8.6          4.44e-16     ***         \n",
      "s(8)                              [0.6]                20           3.6          5.96e-03     **          \n",
      "s(9)                              [0.6]                20           3.6          1.33e-09     ***         \n",
      "s(10)                             [0.6]                20           1.7          3.26e-03     **          \n",
      "s(11)                             [0.6]                20           6.4          6.25e-02     .           \n",
      "s(12)                             [0.6]                20           6.6          1.11e-16     ***         \n",
      "intercept                                              1            0.0          2.23e-13     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-12-5b05aa0e713d>:3: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam.summary()\n"
     ]
    }
   ],
   "source": [
    "from pygam import LinearGAM\n",
    "gam = LinearGAM().fit(boston_data[boston.feature_names], y)\n",
    "gam.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - **回归树：**                                   \n",
    "   基于树的回归方法主要是依据分层和分割的方式将特征空间划分为一系列简单的区域。对某个给定的待预测的自变量，用他所属区域中训练集的平均数或者众数对其进行预测。由于划分特征空间的分裂规则可以用树的形式进行概括，因此这类方法称为决策树方法。决策树由结点(node)和有向边(diredcted edge)组成。结点有两种类型：内部结点(internal node)和叶结点(leaf node)。内部结点表示一个特征或属性，叶结点表示一个类别或者某个值。区域$R_1,R_2$等称为叶节点，将特征空间分开的点为内部节点。                           \n",
    "   ![jupyter](./1.7.png)                     \n",
    "   建立回归树的过程大致可以分为以下两步：          \n",
    "      - 将自变量的特征空间(即$x^{(1)},x^{(2)},x^{(3)},...,x^{(p)}$)的可能取值构成的集合分割成J个互不重叠的区域$R_1,R_2,...,R_j$。        \n",
    "      - 对落入区域$R_j$的每个观测值作相同的预测，预测值等于$R_j$上训练集的因变量的简单算术平均。              \n",
    "   具体来说，就是：             \n",
    "      a.  选择最优切分特征j以及该特征上的最优点s：                \n",
    "      遍历特征j以及固定j后遍历切分点s，选择使得下式最小的(j,s)  $min_{j,s}[min_{c_1}\\sum\\limits_{x_i\\in R_1(j,s)}(y_i-c_1)^2 + min_{c_2}\\sum\\limits_{x_i\\in R_2(j,s)}(y_i-c_2)^2 ]$                           \n",
    "       b. 按照(j,s)分裂特征空间：$R_1(j,s) = \\{x|x^{j} \\le s \\}和R_2(j,s) = \\{x|x^{j} > s \\},\\hat{c}_m = \\frac{1}{N_m}\\sum\\limits_{x \\in R_m(j,s)}y_i,\\;m=1,2$                           \n",
    "       c. 继续调用步骤1，2直到满足停止条件，就是每个区域的样本数小于等于5。        \n",
    "       d. 将特征空间划分为J个不同的区域，生成回归树：$f(x) = \\sum\\limits_{m=1}^{J}\\hat{c}_mI(x \\in R_m)$                \n",
    "       如以下生成的关于运动员在棒球大联盟数据的回归树：             \n",
    "       ![jupyter](./1.8.png)                   \n",
    "    回归树与线性模型的比较：              \n",
    "    线性模型的模型形式与树模型的模型形式有着本质的区别，具体而言，线性回归对模型形式做了如下假定：$f(x) = w_0 + \\sum\\limits_{j=1}^{p}w_jx^{(j)}$，而回归树则是$f(x) = \\sum\\limits_{m=1}^{J}\\hat{c}_mI(x \\in R_m)$。那问题来了，哪种模型更优呢？这个要视具体情况而言，如果特征变量与因变量的关系能很好的用线性关系来表达，那么线性回归通常有着不错的预测效果，拟合效果则优于不能揭示线性结构的回归树。反之，如果特征变量与因变量的关系呈现高度复杂的非线性，那么树方法比传统方法更优。                     \n",
    "    ![jupyter](./1.9.1.png)                        \n",
    "    树模型的优缺点：                 \n",
    "    - 树模型的解释性强，在解释性方面可能比线性回归还要方便。\n",
    "    - 树模型更接近人的决策方式。\n",
    "    - 树模型可以用图来表示，非专业人士也可以轻松解读。\n",
    "    - 树模型可以直接做定性的特征而不需要像线性回归一样哑元化。\n",
    "    - 树模型能很好处理缺失值和异常值，对异常值不敏感，但是这个对线性模型来说却是致命的。\n",
    "    - 树模型的预测准确性一般无法达到其他回归模型的水平，但是改进的方法很多。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "sklearn使用回归树的实例：                \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html?highlight=tree#sklearn.tree.DecisionTreeRegressor                         \n",
    "sklearn.tree.DecisionTreeRegressor(*, criterion='mse', splitter='best', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=None, random_state=None, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, presort='deprecated', ccp_alpha=0.0）                                                 \n",
    "   - 参数：(列举几个重要的，常用的，详情请看上面的官网)                 \n",
    "   criterion：{“ mse”，“ friedman_mse”，“ mae”}，默认=“ mse”。衡量分割标准的函数 。                      \n",
    "   splitter：{“best”, “random”}, default=”best”。分割方式。                    \n",
    "   max_depth：树的最大深度。               \n",
    "   min_samples_split：拆分内部节点所需的最少样本数，默认是2。                      \n",
    "   min_samples_leaf：在叶节点处需要的最小样本数。默认是1。                    \n",
    "   min_weight_fraction_leaf：在所有叶节点处（所有输入样本）的权重总和中的最小加权分数。如果未提供sample_weight，则样本的权重相等。默认是0。                      "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:55:05.005502Z",
     "start_time": "2021-03-14T11:55:04.961503Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "0.9376307599929274"
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeRegressor    \n",
    "reg_tree = DecisionTreeRegressor(criterion = \"mse\",min_samples_leaf = 5)\n",
    "reg_tree.fit(X,y)\n",
    "reg_tree.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 支持向量机回归(SVR)                                        \n",
    "   在介绍支持向量回归SVR之前，我们先来了解下约束优化的相关知识：          \n",
    "      - 约束优化问题(P)：                   \n",
    "      $$\n",
    "      min f(x)  \\\\\n",
    "      s.t.\\;\\;\\;g_i(x) \\le 0,\\; i=1,2,...,m\\\\\n",
    "      \\;\\;\\;\\;\\; h_j(x) = 0,\\; j=1,2,...,l\n",
    "      $$                         \n",
    "   我们假设$x^*$为满足以上条件的局部最优解，$p^* = f(x^*)$，我们的目的就是要找到$x^*$与$p^*$，满足不等式和等式约束的x集合成为可行域，记作S。\n",
    "   \n",
    "    - KKT条件(最优解的一阶必要条件)          \n",
    "    因为KKT条件是最优化的相关内容，在本次开源学习中并不是重点，因此在这里我用一个更加简单的例子说明KKT条件，严格的证明请参见凸优化相关书籍。                       \n",
    "    在这个例子中，我们考虑：($x^*$为我们的最优解)               \n",
    "    $$\n",
    "    minf(x)\\\\\n",
    "    s.t.\\;g_1(x) \\le 0,\\;x \\in R^n\\\\\n",
    "    \\;\\;\\;g_2(x) \\le 0\\\\\n",
    "    \\;\\;\\;g_3(x) \\le 0\n",
    "    $$\n",
    "    ![jupyter](./1.11.png)\n",
    "      \n",
    "      我们可以看到：$-\\nabla f(x^*)$可以由$\\nabla g_1(x^*)$与$\\nabla g_2(x^*)$线性表出，因此有：$-\\nabla f(x^*) = \\lambda_1 \\nabla g_1(x^*) + \\lambda_2 \\nabla g_2(x^*)$，其中$\\lambda_1,\\lambda_2 \\ge 0$，即：                          \n",
    "      $$\n",
    "      \\nabla f(x^*) + \\lambda_1 \\nabla g_1(x^*) + \\lambda_2 \\nabla g_2(x^*) = 0,\\;\\;\\;其中\\lambda_1,\\lambda_2 \\ge 0\n",
    "      $$                       \n",
    "      我们把没有起作用的约束$g_3(x)$也放到式子里面去，目的也就是为了书写方便，即要求：                  \n",
    "      $$\n",
    "      \\nabla f(x^*) + \\lambda_1 \\nabla g_1(x^*) + \\lambda_2 \\nabla g_2(x^*) + \\lambda_3 \\nabla g_3(x^*)= 0,\\;\\;\\;其中\\lambda_1,\\lambda_2 \\ge 0,\\lambda_3 = 0\n",
    "      $$                      \n",
    "      由于点$x^*$位于方程$g_1(x)=0$与$g_2(x)=0$上，因此：$\\lambda_1 g_1(x^*)  = 0,\\lambda_2  g_2(x^*) = 0 , \\lambda_3  g_3(x^*)= 0$                        \n",
    "      \n",
    "      因此，KKT条件就是：假设$x^*$为最优化问题(P)的局部最优解，且$x^*$ 在某个适当的条件下 ,有：                             \n",
    "      $$\n",
    "     \\nabla f(x^*) + \\sum\\limits_{i=1}^{m}\\lambda_i \\nabla g(x^*) + \\sum\\limits_{j=1}^{l}\\mu_j \\nabla h_j(x^*) = 0(对偶条件)\\\\     \n",
    "      \\lambda_i \\ge 0,\\;i = 1,2,...,m(对偶条件)\\\\\n",
    "      g_i(x^*) \\le 0(原问题条件)\\\\\n",
    "      h_j(x^*) = 0(原问题条件)\\\\\n",
    "      \\lambda_i g(x^*) = 0(互补松弛定理)\n",
    "      $$                              \n",
    " - 对偶理论：               \n",
    "   为什么要引入对偶问题呢？是因为原问题与对偶问题就像是一个问题两个角度去看，如利润最大与成本最低等。有时侯原问题上难以解决，但是在对偶问题上就会变得很简单。再者，任何一个原问题在变成对偶问题后都会变成一个凸优化的问题，这点我们后面会有介绍。下面我们来引入对偶问题：                           \n",
    "   首先，我们的原问题(P)是：\n",
    "      $$\n",
    "      min f(x)  \\\\\n",
    "      s.t.\\;\\;\\;g_i(x) \\le 0,\\; i=1,2,...,m\\\\\n",
    "      \\;\\;\\;\\;\\; h_j(x) = 0,\\; j=1,2,...,l\n",
    "      $$                             \n",
    "      引入拉格朗日函数：$L(x,\\lambda,\\mu) = f(x) + \\sum\\limits_{i=1}^{m}\\lambda_i g_i(x) + \\sum\\limits_{j=1}^{l}\\mu_j h_j(x)$                        \n",
    "      拉格朗日对偶函数：                                  \n",
    "      $$\n",
    "      d(\\lambda,\\mu)  =  min_{x\\in X}\\{ f(x) + \\sum\\limits_{i=1}^{m}\\lambda_i g_i(x) + \\sum\\limits_{j=1}^{l}\\mu_j h_j(x)\\} ,其中X为满足条件的x变量\\\\\n",
    "      \\le min_{x\\in S}\\{ f(x) + \\sum\\limits_{i=1}^{m}\\lambda_i g_i(x) + \\sum\\limits_{j=1}^{l}\\mu_j h_j(x) \\},由于g_i(x) \\le 0,h_j(x) = 0,\\lambda_i \\ge 0 ,其中S为可行域\\\\\n",
    "      \\le min_{x\\in S}\\{f(x) \\}\n",
    "      $$                                   \n",
    "      因此：拉格朗日对偶函数$d(\\lambda,\\mu)$是原问题最优解的函数值$p^*$的下界，即每个不同的$\\lambda$与$\\mu$确定的$d(\\lambda,\\mu)$都是$p^*$的下界，但是我们希望下界越大越好，因为越大就更能接近真实的$p^*$。因此：                               \n",
    "      拉格朗日对偶问题(D)转化为：                 \n",
    "      $$\n",
    "      max_{\\lambda,\\mu}d(\\lambda,\\mu)\\\\\n",
    "      s.t. \\lambda_i \\ge 0,i = 1,2,...,m\\\\\n",
    "      也就是：\\\\\n",
    "      max_{\\lambda \\ge 0,\\mu}\\;min_{x \\in S} L(x,\\lambda,\\mu)\n",
    "      $$                   \n",
    "      我们可以观察到，对偶问题是关于$\\lambda$和$\\mu$的线性函数，因此对偶问题是一个凸优化问题，凸优化问题在最优化理论较为简单。\n",
    "      弱对偶定理：对偶问题(D)的最优解$D^*$一定小于原问题最优解$P^*$，这点在刚刚的讨论得到了充分的证明，一定成立。                   \n",
    "      强对偶定理：对偶问题(D)的最优解$D^*$在一定的条件下等于原问题最优解$P^*$，条件非常多样化且不是唯一的，也就是说这是个开放性的问题，在这里我给出一个最简单的条件，即：$f(x)$与$g_i(x)$为凸函数，$h_j(x)$为线性函数，X是凸集，$x^*$满足KKT条件，那么$D^* = P^*$。 \n",
    "      \n",
    "   \n",
    "      "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 支持向量回归SVR                    \n",
    "   在介绍完了相关的优化知识以后，我们开始正式学习支持向量回归SVR。                    \n",
    "   ![jupyter](./1.10.png)                         \n",
    "   在线性回归的理论中，每个样本点都要计算平方损失，但是SVR却是不一样的。SVR认为：落在$f(x)$的$\\epsilon$邻域空间中的样本点不需要计算损失，这些都是预测正确的，其余的落在$\\epsilon$邻域空间以外的样本才需要计算损失，因此：              \n",
    "   ![jupyter](./1.12.png)                                      \n",
    "   $$\n",
    "   min_{w,b,\\xi_i,\\hat{\\xi}_i} \\frac{1}{2}||w||^2 +C \\sum\\limits_{i=1}^{N}(\\xi_i,\\hat{\\xi}_i)\\\\\n",
    "   s.t.\\;\\;\\; f(x_i) - y_i \\le \\epsilon + \\xi_i\\\\\n",
    "   \\;\\;\\;\\;\\;y_i - f(x_i) \\le  \\epsilon +\\hat{\\xi}_i\\\\\n",
    "   \\;\\;\\;\\;\\; \\xi_i,\\hat{\\xi}_i \\le 0,i = 1,2,...,N\n",
    "   $$                                 \n",
    "   引入拉格朗日函数：                  \n",
    "   $$\n",
    "   \\begin{array}{l}\n",
    "L(w, b, \\alpha, \\hat{\\alpha}, \\xi, \\xi, \\mu, \\hat{\\mu}) \\\\\n",
    "\\quad=\\frac{1}{2}\\|w\\|^{2}+C \\sum_{i=1}^{N}\\left(\\xi_{i}+\\widehat{\\xi}_{i}\\right)-\\sum_{i=1}^{N} \\xi_{i} \\mu_{i}-\\sum_{i=1}^{N} \\widehat{\\xi}_{i} \\widehat{\\mu}_{i} \\\\\n",
    "\\quad+\\sum_{i=1}^{N} \\alpha_{i}\\left(f\\left(x_{i}\\right)-y_{i}-\\epsilon-\\xi_{i}\\right)+\\sum_{i=1}^{N} \\widehat{\\alpha}_{i}\\left(y_{i}-f\\left(x_{i}\\right)-\\epsilon-\\widehat{\\xi}_{i}\\right)\n",
    "\\end{array}\n",
    "   $$                       \n",
    "   再令$L(w, b, \\alpha, \\hat{\\alpha}, \\xi, \\xi, \\mu, \\hat{\\mu})$对$w,b,\\xi,\\hat{\\xi}$求偏导等于0，得： $w=\\sum_{i=1}^{N}\\left(\\widehat{\\alpha}_{i}-\\alpha_{i}\\right) x_{i}$。                             \n",
    "   上述过程中需满足KKT条件，即要求：                 \n",
    "   $$\n",
    "   \\left\\{\\begin{array}{c}\n",
    "\\alpha_{i}\\left(f\\left(x_{i}\\right)-y_{i}-\\epsilon-\\xi_{i}\\right)=0 \\\\\n",
    "\\hat{\\alpha_{i}}\\left(y_{i}-f\\left(x_{i}\\right)-\\epsilon-\\hat{\\xi}_{i}\\right)=0 \\\\\n",
    "\\alpha_{i} \\widehat{\\alpha}_{i}=0, \\xi_{i} \\hat{\\xi}_{i}=0 \\\\\n",
    "\\left(C-\\alpha_{i}\\right) \\xi_{i}=0,\\left(C-\\widehat{\\alpha}_{i}\\right) \\hat{\\xi}_{i}=0\n",
    "\\end{array}\\right.\n",
    "   $$                         \n",
    "   SVR的解形如:$f(x)=\\sum_{i=1}^{N}\\left(\\widehat{\\alpha}_{i}-\\alpha_{i}\\right) x_{i}^{T} x+b$                                                  \n",
    "   \n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "sklearn中使用SVR实例：                        \n",
    "sklearn.svm.SVR(*, kernel='rbf', degree=3, gamma='scale', coef0=0.0, tol=0.001, C=1.0, epsilon=0.1, shrinking=True, cache_size=200, verbose=False, max_iter=-1)            \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVR.html?highlight=svr#sklearn.svm.SVR                        \n",
    "   - 参数：                   \n",
    "   kernel：核函数，{‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’}, 默认=’rbf’。(后面会详细介绍)              \n",
    "   degree：多项式核函数的阶数。默认 = 3。                          \n",
    "   C：正则化参数，默认=1.0。(后面会详细介绍)                            \n",
    "   epsilon：SVR模型允许的不计算误差的邻域大小。默认0.1。              \n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T11:56:00.141392Z",
     "start_time": "2021-03-14T11:56:00.070391Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "0.7024525421955277"
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.svm import SVR\n",
    "from sklearn.preprocessing import StandardScaler     # 标准化数据\n",
    "from sklearn.pipeline import make_pipeline   # 使用管道，把预处理和模型形成一个流程\n",
    "\n",
    "reg_svr = make_pipeline(StandardScaler(), SVR(C=1.0, epsilon=0.2))\n",
    "reg_svr.fit(X, y)\n",
    "reg_svr.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(4) 优化基础模型                   \n",
    "在刚刚的回归问题的基本算法中，我们使用数据集去估计模型的参数，如线性回归模型中的参数w，那么这个数据集我们称为训练数据集，简称训练集。我们在回归问题中使用训练集估计模型的参数的原则一般都是使得我们的损失函数在训练集达到最小值，其实在实际问题中我们是可以让损失函数在训练集最小化为0，如：在线性回归中，我加入非常多的高次项，使得我们模型在训练集的每一个数据点都恰好位于曲线上，那这时候模型在训练集的损失值也就是误差为0。                \n",
    "![jupyter](./1.13.png)                            \n",
    "既然能做到这件事，是不是代表我们的建模完事大吉呢？换句话说我们的模型可以预测任意情况呢？答案是显然否定的。我们建立机器学习的目的并不是为了在已有的数据集，也就是训练集上效果表现非常优异，我们希望建立的机器学习模型在未知且情况复杂的测试数据上表现优异，我们称这样的未出现在训练集的未知数据集成为测试数据集，简称测试集。我们希望模型在测试集上表现优异！因为假如我们根据股票市场前六个月的数据拟合一个预测模型，我们的目的不是为了预测以前这六个月越准越好，而是预测明天乃至未来的股价变化。                                  \n",
    "   - (a) 训练均方误差与测试均方误差：                   \n",
    "   在回归中，我们最常用的评价指标为均方误差，即：$MSE = \\frac{1}{N}\\sum\\limits_{i=1}^{N}(y_i -\\hat{ f}(x_i))^2$，其中$\\hat{ f}(x_i)$是样本$x_i$应用建立的模型$\\hat{f}$预测的结果。如果我们所用的数据是训练集上的数据，那么这个误差为训练均方误差，如果我们使用测试集的数据计算的均方误差，我们称为测试均方误差。一般而言，我们并不关心模型在训练集上的训练均方误差，我们关心的是模型面对未知的样本集，即测试集上的测试误差，我们的目标是使得我们建立的模型在测试集上的测试误差最小。那我们如何选择一个测试误差最小的模型呢？这是个棘手的问题，因为在模型建立阶段，我们是不能得到测试数据的，比如：我们在模型未上线之前是不能拿到未知且真实的测试数据来验证我们的模型的。在这种情况下，为了简便起见，一些观点认为通过训练误差最小化来选择模型也是可行的。这种观点表面看上去是可行的，但是存在一个致命的缺点，那就是：一个模型的训练均方误差最小时，不能保证测试均方误差同时也很小。对于这种想法构造的模型，一般在训练误差达到最小时，测试均方误差一般很大！如图：                       \n",
    "   ![jupyter](./1.14.png)                       \n",
    "   ![jupyter](./1.15.png)                    \n",
    "   可以看到：当我们的模型的训练均方误差达到很小时，测试均方误差反而很大，但是我们寻找的最优的模型是测试均方误差达到最小时对应的模型，因此基于训练均方误差达到最小选择模型本质上是行不同的。正如上右图所示：模型在训练误差很小，但是测试均方误差很大时，我们称这种情况叫模型的**过拟合**。                            \n",
    "   - (b) 偏差-方差的权衡：                         \n",
    "   从上图的测试均方误差曲线可以看到：测试均方误差曲线呈现U型曲线，这表明了在测试误差曲线中有两种力量在互相博弈。可以证明：           \n",
    "   $$\n",
    "   E\\left(y_{0}-\\hat{f}\\left(x_{0}\\right)\\right)^{2}=\\operatorname{Var}\\left(\\hat{f}\\left(x_{0}\\right)\\right)+\\left[\\operatorname{Bias}\\left(\\hat{f}\\left(x_{0}\\right)\\right)\\right]^{2}+\\operatorname{Var}(\\varepsilon)\n",
    "   $$                  \n",
    "   也就是说，我们的测试均方误差的期望值可以分解为$\\hat{f}(x_0)$的方差、$\\hat{f}(x_0)$的偏差平方和误差项$\\epsilon$的方差。为了使得模型的测试均方误差达到最小值，也就是同时最小化偏差的平方和方差。由于我们知道偏差平方和方差本身是非负的，因此测试均方误差的期望不可能会低于误差的方差，因此我们称$\\operatorname{Var}(\\varepsilon)$为建模任务的难度，这个量在我们的任务确定后是无法改变的，也叫做不可约误差。那么模型的方差和偏差的平方和究竟是什么呢？所谓模型的方差就是：用不同的数据集去估计$f$时，估计函数的改变量。举个例子：我们想要建立一个线性回归模型，可以通过输入中国人身高去预测我们的体重。但是显然我们没有办法把全中国13亿人做一次人口普查，拿到13亿人的身高体重去建立模型。我们能做的就是从13亿中抽1000个样本进行建模，我们对这个抽样的过程重复100遍，就会得到100个1000人的样本集。我们使用线性回归模型估计参数就能得到100个线性回归模型。由于样本抽取具有随机性，我们得到的100个模型不可能参数完全一样，那么这100个模型之间的差异就叫做方差。显然，我们希望得到一个稳定的模型，也就是在不同的样本集估计的模型都不会相差太大，即要求f的方差越小越好。**一般来说，模型的复杂度越高，f的方差就会越大。** 如加入二次项的模型的方差比线性回归模型的方差要大。                              \n",
    "   ![jupyter](./1.16.png)                                \n",
    "   另一方面，模型的偏差是指：为了选择一个简单的模型去估计真实函数所带入的误差。假如真实的数据X与Y的关系是二次关系，但是我们选择了线性模型进行建模，那由于模型的复杂度引起的这种误差我们称为偏差，它的构成时复杂的。偏差度量了学习算法的期望预测与真实结果的偏离程度，即刻画了学习算法本身的拟合能力。偏差度量的是单个模型的学习能力，而方差度量的是同一个模型在不同数据集上的稳定性。“偏差-方差分解”说明：泛化性能是由学习算法的能力、数据的充分性以及学习任务本身的难度所共同决定的。给定学习任务，为了取得好的泛化性能，则需使偏差较小，即能够充分拟合数据，并且使方差较小，即使得数据扰动产生的影响小。                          \n",
    "   ![jupyter](./1.17.png)                    \n",
    "   一般而言，增加模型的复杂度，会增加模型的方差，但是会减少模型的偏差，我们要找到一个方差--偏差的权衡，使得测试均方误差最。                        \n",
    "   ![jupyter](./1.18.png)                              \n",
    "   (c) 特征提取：              \n",
    "   在前面的讨论中，我们已经明确一个目标，就是：我们要选择一个测试误差达到最小的模型。但是实际上我们很难对实际的测试误差做精确的计算，因此我们要对测试误差进行估计，估计的方式有两种：训练误差修正与交叉验证。                         \n",
    "      - 训练误差修正：      \n",
    "      前面的讨论我们已经知道，模型越复杂，训练误差越小，测试误差先减后增。因此，我们先构造一个特征较多的模型使其过拟合，此时训练误差很小而测试误差很大，那这时我们加入关于特征个数的惩罚。因此，当我们的训练误差随着特征个数的增加而减少时，惩罚项因为特征数量的增加而增大，抑制了训练误差随着特征个数的增加而无休止地减小。具体的数学量如下：             \n",
    "      $C_p = \\frac{1}{N}(RSS  +  2d\\hat{\\sigma}^2)$，其中d为模型特征个数，$RSS = \\sum\\limits_{i=1}^{N}(y_i-\\hat{f}(x_i))^2$，$\\hat{\\sigma}^2$为模型预测误差的方差的估计值，即残差的方差。                       \n",
    "      AIC赤池信息量准则：$AIC = \\frac{1}{d\\hat{\\sigma}^2}(RSS  +  2d\\hat{\\sigma}^2)$                              \n",
    "      BIC贝叶斯信息量准则：$BIC = \\frac{1}{n}(RSS + log(n)d\\hat{\\sigma}^2)$                     \n",
    "      - 交叉验证：                   \n",
    "      前面讨论的对训练误差修正得到测试误差的估计是间接方法，这种方法的桥梁是训练误差，而交叉验证则是对测试误差的直接估计。交叉验证比训练误差修正的优势在于：能够给出测试误差的一个直接估计。在这里只介绍K折交叉验证：我们把训练样本分成K等分，然后用K-1个样本集当做训练集，剩下的一份样本集为验证集去估计由K-1个样本集得到的模型的精度，这个过程重复K次取平均值得到测试误差的一个估计$CV_{(K)} = \\frac{1}{K}\\sum\\limits_{i=1}^{K}MSE_i$。5折交叉验证如下图：（蓝色的是训练集，黄色的是验证集）                     \n",
    "      ![jupyter](./1.19.png)                   \n",
    "   \n",
    "   在测试误差能够被合理的估计出来以后，我们做特征选择的目标就是：从p个特征中选择m个特征，使得对应的模型的测试误差的估计最小。对应的方法有：\n",
    "      - 最优子集选择：                  \n",
    "      (i) 记不含任何特征的模型为$M_0$，计算这个$M_0$的测试误差。                                              \n",
    "      (ii) 在$M_0$基础上增加一个变量，计算p个模型的RSS，选择RSS最小的模型记作$M_1$，并计算该模型$M_1$的测试误差。                                          \n",
    "      (iii) 再增加变量，计算p-1个模型的RSS，并选择RSS最小的模型记作$M_2$，并计算该模型$M_2$的测试误差。                                                \n",
    "      (iv) 重复以上过程知道拟合的模型有p个特征为止，并选择p+1个模型$\\{M_0,M_1,...,M_p \\}$中测试误差最小的模型作为最优模型。                           \n",
    "      - 向前逐步选择：               \n",
    "      最优子集选择虽然在原理上很直观，但是随着数据特征维度p的增加，子集的数量为$2^p$，计算效率非常低下且需要的计算内存也很高，在大数据的背景下显然不适用。因此，我们需要把最优子集选择的运算效率提高，因此向前逐步选择算法的过程如下：                            \n",
    "      (i) 记不含任何特征的模型为$M_0$，计算这个$M_0$的测试误差。                    \n",
    "      (ii) 在$M_0$基础上增加一个变量，计算p个模型的RSS，选择RSS最小的模型记作$M_1$，并计算该模型$M_1$的测试误差。                  \n",
    "      (iii) 在最小的RSS模型下继续增加一个变量，选择RSS最小的模型记作$M_2$，并计算该模型$M_2$的测试误差。                  \n",
    "      (iv) 以此类推，重复以上过程知道拟合的模型有p个特征为止，并选择p+1个模型$\\{M_0,M_1,...,M_p \\}$中测试误差最小的模型作为最优模型。                      \n",
    "  \n",
    "  (d) 压缩估计(正则化)：                         \n",
    "  除了刚刚讨论的直接对特征自身进行选择以外，我们还可以对回归的系数进行约束或者加罚的技巧对p个特征的模型进行拟合，显著降低模型方差，这样也会提高模型的拟合效果。具体来说，就是将回归系数往零的方向压缩，这也就是为什么叫压缩估计的原因了。                          \n",
    "     - 岭回归(L2正则化的例子)：                 \n",
    "     在线性回归中，我们的损失函数为$J(w) = \\sum\\limits_{i=1}^{N}(y_i-w_0-\\sum\\limits_{j=1}^{p}w_jx_{ij})^2$，我们在线性回归的损失函数的基础上添加对系数的约束或者惩罚，即：                           \n",
    "     $$\n",
    "     J(w) = \\sum\\limits_{i=1}^{N}(y_i-w_0-\\sum\\limits_{j=1}^{p}w_jx_{ij})^2 + \\lambda\\sum\\limits_{j=1}^{p}w_j^2,\\;\\;其中，\\lambda \\ge 0\\\\\n",
    "     \\hat{w} = (X^TX + \\lambda I)^{-1}X^TY\n",
    "     $$                        \n",
    "   调节参数$\\lambda$的大小是影响压缩估计的关键，$\\lambda$越大，惩罚的力度越大，系数则越趋近于0，反之，选择合适的$\\lambda$对模型精度来说十分重要。岭回归通过牺牲线性回归的无偏性降低方差，有可能使得模型整体的测试误差较小，提高模型的泛化能力。                                 \n",
    "     - Lasso回归(L1正则化的例子)：             \n",
    "     岭回归的一个很显著的特点是：将模型的系数往零的方向压缩，但是岭回归的系数只能趋于0但无法等于0，换句话说，就是无法做特征选择。能否使用压缩估计的思想做到像特征最优子集选择那样提取出重要的特征呢？答案是肯定的！我们只需要对岭回归的优化函数做小小的调整就行了，我们使用系数向量的L1范数替换岭回归中的L2范数：                     \n",
    "     $$\n",
    "     J(w) = \\sum\\limits_{i=1}^{N}(y_i-w_0-\\sum\\limits_{j=1}^{p}w_jx_{ij})^2 + \\lambda\\sum\\limits_{j=1}^{p}|w_j|,\\;\\;其中，\\lambda \\ge 0\n",
    "     $$                  \n",
    "   为什么Losso能做到特征选择而岭回归却不能呢个做到呢？(如图：左边为Lasso，右边为岭回归)                \n",
    "   ![jupyter](./1.20.png)                       \n",
    "   椭圆形曲线为RSS等高线，菱形和圆形区域分别代表了L1和L2约束，Lasso回归和岭回归都是在约束下的回归，因此最优的参数为椭圆形曲线与菱形和圆形区域相切的点。但是Lasso回归的约束在每个坐标轴上都有拐角，因此当RSS曲线与坐标轴相交时恰好回归系数中的某一个为0，这样就实现了特征提取。反观岭回归的约束是一个圆域，没有尖点，因此与RSS曲线相交的地方一般不会出现在坐标轴上，因此无法让某个特征的系数为0，因此无法做到特征提取。                                \n",
    "   \n",
    "   (e) 降维：                \n",
    "   到目前为止，我们所讨论的方法对方差的控制有两种方式：一种是使用原始变量的子集，另一种是将变量系数压缩至零。但是这些方法都是基于原始特征$x_1,...,x_p$得到的，现在我们探讨一类新的方法：将原始的特征空间投影到一个低维的空间实现变量的数量变少，如：将二维的平面投影至一维空间。机器学习领域中所谓的降维就是指采用某种映射方法，将原高维空间中的数据点映射到低维度的空间中。降维的本质是学习一个映射函数 f : x->y，其中x是原始数据点的表达，目前最多使用向量表达形式。 y是数据点映射后的低维向量表达，通常y的维度小于x的维度（当然提高维度也是可以的）。f可能是显式的或隐式的、线性的或非线性的。目前大部分降维算法处理向量表达的数据，也有一些降维算法处理高阶张量表达的数据。之所以使用降维后的数据表示是因为在原始的高维空间中，包含有冗余信息以及噪音信息，在实际应用例如图像识别中造成了误差，降低了准确率；而通过降维,我们希望减少 冗余信息 所造成的误差,提高识别（或其他应用）的精度。又或者希望通过降维算法来寻找数据内部的本质结构特征。在很多算法中，降维算法成为了数据预处理的一部分，如PCA。事实上，有一些算法如果没有降维预处理，其实是很难得到很好的效果的。 (摘自：rosenor1博客)                                \n",
    "   主成分分析(PCA)：                     \n",
    "   主成分分析的思想：通过**最大投影方差** 将原始空间进行重构，即由特征相关重构为无关，即落在某个方向上的点(投影)的方差最大。在进行下一步推导之前，我们先把样本均值和样本协方差矩阵推广至矩阵形式：                     \n",
    "   样本均值Mean:$\\bar{x} = \\frac{1}{N}\\sum\\limits_{i=1}^{N}x_i =  \\frac{1}{N}X^T1_N,\\;\\;\\;其中1_N = (1,1,...,1)_{N}^T$                         \n",
    "   样本协方差矩阵$S^2 = \\frac{1}{N}\\sum\\limits_{i=1}^{N}(x_i-\\bar{x})(x_i-\\bar{x})^T = \\frac{1}{N}X^THX,\\;\\;\\;其中，H = I_N - \\frac{1}{N}1_N1_N^T$                             \n",
    "   最大投影方差的步骤：                \n",
    "   (i) 中心化：$x_i - \\bar{x}$                       \n",
    "   (ii) 计算每个点$x_1,...,x_N$至$\\vec{u}_1$方向上的投影：$(x_i-\\bar{x})\\vec{u}_1,\\;\\;\\;||\\vec{u}_1|| = 1$                             \n",
    "   (iii) 计算投影方差：$J = \\frac{1}{N}\\sum\\limits_{i=1}^{N}[(x_i-\\bar{x})^T\\vec{u}_1]^2,\\;\\;\\;||\\vec{u}_1|| = 1$                           \n",
    "   (iv) 最大化投影方差求$\\vec{u}_1$：                       \n",
    "   $$\n",
    "   \\bar{u}_1 = argmax_{u_1}\\;\\;\\frac{1}{N}\\sum\\limits_{i=1}^{N}[(x_i-\\bar{x})^T\\vec{u}_1]^2 \\\\\n",
    "   \\;\\;\\;s.t. \\vec{u}_1^T\\vec{u}_1 = 1 (\\vec{u}_1往后不带向量符号)\n",
    "   $$                         \n",
    "   得到：               \n",
    "   $$\n",
    "   J = \\frac{1}{N}\\sum\\limits_{i=1}^{N}[(x_i-\\bar{x})^T\\vec{u}_1]^2  = \\frac{1}{N}\\sum\\limits_{i=1}^{N}[u_1^T(x_i-\\bar{x})(x_i-\\bar{x})^Tu_1]\\\\\n",
    "   \\; = u_1^T[\\frac{1}{N}\\sum\\limits_{i=1}^{N}(x_i-\\bar{x})(x_i - \\bar{x})^T]u_1 = u_1^TS^2u_1\n",
    "   $$                    \n",
    "   即：           \n",
    "   $$\n",
    "   \\hat{u}_1 = argmax_{u_1}u_1^TS^2u_1,\\;\\;\\;s.t.u_1^Tu_1 = 1\\\\\n",
    "   L(u_1,\\lambda) = u_1^TS^2u_1 + \\lambda (1-u_1^Tu_1)\\\\\n",
    "   \\frac{\\partial L}{\\partial u_1} = 2S^2u_1-2\\lambda u_1 = 0\\\\\n",
    "   即：S^2u_1 = \\lambda u_1\n",
    "   $$                        \n",
    "   可以看到：$\\lambda$为$S^2$的特征值，$u_1$为$S^2$的特征向量。因此我们只需要对中心化后的协方差矩阵进行特征值分解，得到的特征向量即为投影方向。如果需要进行降维，那么只需要取p的前M个特征向量即可。\n",
    "   \n",
    "    \n",
    "   \n",
    "      \n",
    "      \n",
    "   \n",
    "\n",
    "\n",
    "     \n",
    "   \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "特征提取的实例：向前逐步回归                      \n",
    "案例来源：https://blog.csdn.net/weixin_44835596/article/details/89763300                            \n",
    "根据AIC准则定义向前逐步回归进行变量筛选"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:14:19.500400Z",
     "start_time": "2021-03-14T12:14:19.485401Z"
    }
   },
   "outputs": [],
   "source": [
    "#定义向前逐步回归函数\n",
    "def forward_select(data,target):\n",
    "    variate=set(data.columns)  #将字段名转换成字典类型\n",
    "    variate.remove(target)  #去掉因变量的字段名\n",
    "    selected=[]\n",
    "    current_score,best_new_score=float('inf'),float('inf')  #目前的分数和最好分数初始值都为无穷大（因为AIC越小越好）\n",
    "    #循环筛选变量\n",
    "    while variate:\n",
    "        aic_with_variate=[]\n",
    "        for candidate in variate:  #逐个遍历自变量\n",
    "            formula=\"{}~{}\".format(target,\"+\".join(selected+[candidate]))  #将自变量名连接起来\n",
    "            aic=ols(formula=formula,data=data).fit().aic  #利用ols训练模型得出aic值\n",
    "            aic_with_variate.append((aic,candidate))  #将第每一次的aic值放进空列表\n",
    "        aic_with_variate.sort(reverse=True)  #降序排序aic值\n",
    "        best_new_score,best_candidate=aic_with_variate.pop()  #最好的aic值等于删除列表的最后一个值，以及最好的自变量等于列表最后一个自变量\n",
    "        if current_score>best_new_score:  #如果目前的aic值大于最好的aic值\n",
    "            variate.remove(best_candidate)  #移除加进来的变量名，即第二次循环时，不考虑此自变量了\n",
    "            selected.append(best_candidate)  #将此自变量作为加进模型中的自变量\n",
    "            current_score=best_new_score  #最新的分数等于最好的分数\n",
    "            print(\"aic is {},continuing!\".format(current_score))  #输出最小的aic值\n",
    "        else:\n",
    "            print(\"Selection over!\")\n",
    "            break\n",
    "    formula=\"{}~{}\".format(target,\"+\".join(selected))  #最终的模型式子\n",
    "    print(\"Final formula is: {}\".format(formula))\n",
    "    model=ols(formula=formula,data=data).fit()\n",
    "    return(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:14:20.656426Z",
     "start_time": "2021-03-14T12:14:20.112430Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "aic is 3286.974956900157,continuing!\n",
      "aic is 3171.5423142992013,continuing!\n",
      "aic is 3114.0972674193326,continuing!\n",
      "aic is 3097.359044862759,continuing!\n",
      "aic is 3069.438633167217,continuing!\n",
      "aic is 3057.9390497191152,continuing!\n",
      "aic is 3048.438382711162,continuing!\n",
      "aic is 3042.274993098419,continuing!\n",
      "aic is 3040.154562175143,continuing!\n",
      "aic is 3032.0687017003256,continuing!\n",
      "aic is 3021.726387825062,continuing!\n",
      "Selection over!\n",
      "Final formula is: Price~LSTAT+RM+PTRATIO+DIS+NOX+CHAS+B+ZN+CRIM+RAD+TAX\n"
     ]
    },
    {
     "data": {
      "text/plain": "<statsmodels.regression.linear_model.RegressionResultsWrapper at 0x182e2227dc0>"
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import statsmodels.api as sm #最小二乘\n",
    "from statsmodels.formula.api import ols #加载ols模型\n",
    "forward_select(data=boston_data,target=\"Price\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:18:39.748590Z",
     "start_time": "2021-03-14T12:18:39.706590Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "<class 'statsmodels.iolib.summary.Summary'>\n\"\"\"\n                            OLS Regression Results                            \n==============================================================================\nDep. Variable:                  Price   R-squared:                       0.741\nModel:                            OLS   Adj. R-squared:                  0.735\nMethod:                 Least Squares   F-statistic:                     128.2\nDate:                Tue, 16 Mar 2021   Prob (F-statistic):          5.54e-137\nTime:                        18:06:01   Log-Likelihood:                -1498.9\nNo. Observations:                 506   AIC:                             3022.\nDf Residuals:                     494   BIC:                             3072.\nDf Model:                          11                                         \nCovariance Type:            nonrobust                                         \n==============================================================================\n                 coef    std err          t      P>|t|      [0.025      0.975]\n------------------------------------------------------------------------------\nIntercept     36.3411      5.067      7.171      0.000      26.385      46.298\nLSTAT         -0.5226      0.047    -11.019      0.000      -0.616      -0.429\nRM             3.8016      0.406      9.356      0.000       3.003       4.600\nPTRATIO       -0.9465      0.129     -7.334      0.000      -1.200      -0.693\nDIS           -1.4927      0.186     -8.037      0.000      -1.858      -1.128\nNOX          -17.3760      3.535     -4.915      0.000     -24.322     -10.430\nCHAS           2.7187      0.854      3.183      0.002       1.040       4.397\nB              0.0093      0.003      3.475      0.001       0.004       0.015\nZN             0.0458      0.014      3.390      0.001       0.019       0.072\nCRIM          -0.1084      0.033     -3.307      0.001      -0.173      -0.044\nRAD            0.2996      0.063      4.726      0.000       0.175       0.424\nTAX           -0.0118      0.003     -3.493      0.001      -0.018      -0.005\n==============================================================================\nOmnibus:                      178.430   Durbin-Watson:                   1.078\nProb(Omnibus):                  0.000   Jarque-Bera (JB):              787.785\nSkew:                           1.523   Prob(JB):                    8.60e-172\nKurtosis:                       8.300   Cond. No.                     1.47e+04\n==============================================================================\n\nNotes:\n[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n[2] The condition number is large, 1.47e+04. This might indicate that there are\nstrong multicollinearity or other numerical problems.\n\"\"\"",
      "text/html": "<table class=\"simpletable\">\n<caption>OLS Regression Results</caption>\n<tr>\n  <th>Dep. Variable:</th>          <td>Price</td>      <th>  R-squared:         </th> <td>   0.741</td> \n</tr>\n<tr>\n  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.735</td> \n</tr>\n<tr>\n  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   128.2</td> \n</tr>\n<tr>\n  <th>Date:</th>             <td>Tue, 16 Mar 2021</td> <th>  Prob (F-statistic):</th> <td>5.54e-137</td>\n</tr>\n<tr>\n  <th>Time:</th>                 <td>18:06:01</td>     <th>  Log-Likelihood:    </th> <td> -1498.9</td> \n</tr>\n<tr>\n  <th>No. Observations:</th>      <td>   506</td>      <th>  AIC:               </th> <td>   3022.</td> \n</tr>\n<tr>\n  <th>Df Residuals:</th>          <td>   494</td>      <th>  BIC:               </th> <td>   3072.</td> \n</tr>\n<tr>\n  <th>Df Model:</th>              <td>    11</td>      <th>                     </th>     <td> </td>    \n</tr>\n<tr>\n  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>    \n</tr>\n</table>\n<table class=\"simpletable\">\n<tr>\n      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n</tr>\n<tr>\n  <th>Intercept</th> <td>   36.3411</td> <td>    5.067</td> <td>    7.171</td> <td> 0.000</td> <td>   26.385</td> <td>   46.298</td>\n</tr>\n<tr>\n  <th>LSTAT</th>     <td>   -0.5226</td> <td>    0.047</td> <td>  -11.019</td> <td> 0.000</td> <td>   -0.616</td> <td>   -0.429</td>\n</tr>\n<tr>\n  <th>RM</th>        <td>    3.8016</td> <td>    0.406</td> <td>    9.356</td> <td> 0.000</td> <td>    3.003</td> <td>    4.600</td>\n</tr>\n<tr>\n  <th>PTRATIO</th>   <td>   -0.9465</td> <td>    0.129</td> <td>   -7.334</td> <td> 0.000</td> <td>   -1.200</td> <td>   -0.693</td>\n</tr>\n<tr>\n  <th>DIS</th>       <td>   -1.4927</td> <td>    0.186</td> <td>   -8.037</td> <td> 0.000</td> <td>   -1.858</td> <td>   -1.128</td>\n</tr>\n<tr>\n  <th>NOX</th>       <td>  -17.3760</td> <td>    3.535</td> <td>   -4.915</td> <td> 0.000</td> <td>  -24.322</td> <td>  -10.430</td>\n</tr>\n<tr>\n  <th>CHAS</th>      <td>    2.7187</td> <td>    0.854</td> <td>    3.183</td> <td> 0.002</td> <td>    1.040</td> <td>    4.397</td>\n</tr>\n<tr>\n  <th>B</th>         <td>    0.0093</td> <td>    0.003</td> <td>    3.475</td> <td> 0.001</td> <td>    0.004</td> <td>    0.015</td>\n</tr>\n<tr>\n  <th>ZN</th>        <td>    0.0458</td> <td>    0.014</td> <td>    3.390</td> <td> 0.001</td> <td>    0.019</td> <td>    0.072</td>\n</tr>\n<tr>\n  <th>CRIM</th>      <td>   -0.1084</td> <td>    0.033</td> <td>   -3.307</td> <td> 0.001</td> <td>   -0.173</td> <td>   -0.044</td>\n</tr>\n<tr>\n  <th>RAD</th>       <td>    0.2996</td> <td>    0.063</td> <td>    4.726</td> <td> 0.000</td> <td>    0.175</td> <td>    0.424</td>\n</tr>\n<tr>\n  <th>TAX</th>       <td>   -0.0118</td> <td>    0.003</td> <td>   -3.493</td> <td> 0.001</td> <td>   -0.018</td> <td>   -0.005</td>\n</tr>\n</table>\n<table class=\"simpletable\">\n<tr>\n  <th>Omnibus:</th>       <td>178.430</td> <th>  Durbin-Watson:     </th> <td>   1.078</td> \n</tr>\n<tr>\n  <th>Prob(Omnibus):</th> <td> 0.000</td>  <th>  Jarque-Bera (JB):  </th> <td> 787.785</td> \n</tr>\n<tr>\n  <th>Skew:</th>          <td> 1.523</td>  <th>  Prob(JB):          </th> <td>8.60e-172</td>\n</tr>\n<tr>\n  <th>Kurtosis:</th>      <td> 8.300</td>  <th>  Cond. No.          </th> <td>1.47e+04</td> \n</tr>\n</table><br/><br/>Notes:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.47e+04. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lm=ols(\"Price~LSTAT+RM+PTRATIO+DIS+NOX+CHAS+B+ZN+CRIM+RAD+TAX\",data=boston_data).fit()\n",
    "lm.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "岭回归实例分享：                   \n",
    "sklearn.linear_model.ridge_regression(X, y, alpha, *, sample_weight=None, solver='auto', max_iter=None, tol=0.001, verbose=0, random_state=None, return_n_iter=False, return_intercept=False, check_input=True)                          \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.ridge_regression.html?highlight=rid#sklearn.linear_model.ridge_regression                      \n",
    "   - 参数：                     \n",
    "   alpha：较大的值表示更强的正则化。浮点数                   \n",
    "   sample_weight：样本权重，默认无。                      \n",
    "   solver：求解方法，{‘auto’, ‘svd’, ‘cholesky’, ‘lsqr’, ‘sparse_cg’, ‘sag’, ‘saga’}, 默认=’auto’。“ svd”使用X的奇异值分解来计算Ridge系数。'cholesky'使用标准的scipy.linalg.solve函数通过dot（XT，X）的Cholesky分解获得封闭形式的解。'sparse_cg'使用scipy.sparse.linalg.cg中的共轭梯度求解器。作为一种迭代算法，对于大规模数据（可能设置tol和max_iter），此求解器比“ Cholesky”更合适。 lsqr”使用专用的正则化最小二乘例程scipy.sparse.linalg.lsqr。它是最快的，并且使用迭代过程。“ sag”使用随机平均梯度下降，“ saga”使用其改进的无偏版本SAGA。两种方法都使用迭代过程，并且当n_samples和n_features都很大时，通常比其他求解器更快。请注意，只有在比例大致相同的要素上才能确保“ sag”和“ saga”快速收敛。您可以使用sklearn.preprocessing中的缩放器对数据进行预处理。最后五个求解器均支持密集和稀疏数据。但是，当fit_intercept为True时，仅'sag'和'sparse_cg'支持稀疏输入。                             \n",
    "   \n",
    "   \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": "0.739957023371629"
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import linear_model\n",
    "reg_rid = linear_model.Ridge(alpha=.5)\n",
    "reg_rid.fit(X,y)\n",
    "reg_rid.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lasso实例分享：                  \n",
    "class sklearn.linear_model.Lasso(alpha=1.0, *, fit_intercept=True, normalize=False, precompute=False, copy_X=True, max_iter=1000, tol=0.0001, warm_start=False, positive=False, random_state=None, selection='cyclic')                           \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html?highlight=lasso#sklearn.linear_model.Lasso                                \n",
    "   - 参数：                \n",
    "   alpha：正则化强度，1.0代表标准最小二乘。                \n",
    "   fit_intercept：是否计算模型截距。默认true。                         \n",
    "   normalize：是否标准化，默认false。                   \n",
    "   positive：是否强制系数为正，默认false。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": "0.7140164719858566"
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import linear_model\n",
    "reg_lasso = linear_model.Lasso(alpha = 0.5)\n",
    "reg_lasso.fit(X,y)\n",
    "reg_lasso.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(5) 对模型超参数进行调优(调参)：                             \n",
    "在刚刚的讨论中，我们似乎对模型的优化都是对模型算法本身的改进，比如：岭回归对线性回归的优化在于在线性回归的损失函数中加入L2正则化项从而牺牲无偏性降低方差。但是，大家是否想过这样的问题：在L2正则化中参数$\\lambda$应该选择多少？是0.01、0.1、还是1？到目前为止，我们只能凭经验或者瞎猜，能不能找到一种方法找到最优的参数$\\lambda$？事实上，找到最佳参数的问题本质上属于最优化的内容，因为从一个参数集合中找到最佳的值本身就是最优化的任务之一，我们脑海中浮现出来的算法无非就是：梯度下降法、牛顿法等无约束优化算法或者约束优化算法，但是在具体验证这个想法是否可行之前，我们必须先认识两个最本质概念的区别。                                     \n",
    "   - 参数与超参数：                            \n",
    "   我们很自然的问题就是岭回归中的参数$\\lambda$和参数w之间有什么不一样？事实上，参数w是我们通过设定某一个具体的$\\lambda$后使用类似于最小二乘法、梯度下降法等方式优化出来的，我们总是设定了$\\lambda$是多少后才优化出来的参数w。因此，类似于参数w一样，使用最小二乘法或者梯度下降法等最优化算法优化出来的数我们称为参数，类似于$\\lambda$一样，我们无法使用最小二乘法或者梯度下降法等最优化算法优化出来的数我们称为超参数。                                       \n",
    "   模型参数是模型内部的配置变量，其值可以根据数据进行估计。                       \n",
    "      - 进行预测时需要参数。                     \n",
    "      - 它参数定义了可使用的模型。                        \n",
    "      - 参数是从数据估计或获悉的。                       \n",
    "      - 参数通常不由编程者手动设置。                     \n",
    "      - 参数通常被保存为学习模型的一部分。                      \n",
    "      - 参数是机器学习算法的关键，它们通常由过去的训练数据中总结得出 。                          \n",
    "   模型超参数是模型外部的配置，其值无法从数据中估计。\n",
    "      - 超参数通常用于帮助估计模型参数。\n",
    "      - 超参数通常由人工指定。\n",
    "      - 超参数通常可以使用启发式设置。\n",
    "      - 超参数经常被调整为给定的预测建模问题。                            \n",
    "   我们前面(4)部分的优化都是基于模型本身的具体形式的优化，那本次(5)调整的内容是超参数，也就是取不同的超参数的值对于模型的性能有不同的影响。                             \n",
    "   - 网格搜索GridSearchCV()：                \n",
    "   网格搜索：https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html?highlight=gridsearchcv#sklearn.model_selection.GridSearchCV                         \n",
    "   网格搜索结合管道：https://scikit-learn.org/stable/auto_examples/compose/plot_compare_reduction.html?highlight=gridsearchcv                              \n",
    "   网格搜索的思想非常简单，比如你有2个超参数需要去选择，那你就把所有的超参数选择列出来分别做排列组合。举个例子：$\\lambda = 0.01,0.1,1.0$和$\\alpha = 0.01,0.1,1.0$,你可以做一个排列组合，即：{[0.01,0.01],[0.01,0.1],[0.01,1],[0.1,0.01],[0.1,0.1],[0.1,1.0],[1,0.01],[1,0.1],[1,1]}  ，然后针对每组超参数分别建立一个模型，然后选择测试误差最小的那组超参数。换句话说，我们需要从超参数空间中寻找最优的超参数，很像一个网格中找到一个最优的节点，因此叫网格搜索。                         \n",
    "   - 随机搜索 RandomizedSearchCV() ：               \n",
    "   https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html?highlight=randomizedsearchcv#sklearn.model_selection.RandomizedSearchCV                           \n",
    "   网格搜索相当于暴力地从参数空间中每个都尝试一遍，然后选择最优的那组参数，这样的方法显然是不够高效的，因为随着参数类别个数的增加，需要尝试的次数呈指数级增长。有没有一种更加高效的调优方式呢？那就是使用随机搜索的方式，这种方式不仅仅高校，而且实验证明，随机搜索法结果比稀疏化网格法稍好(有时候也会极差，需要权衡)。参数的随机搜索中的每个参数都是从可能的参数值的分布中采样的。与网格搜索相比，这有两个主要优点：        \n",
    "      - 可以独立于参数数量和可能的值来选择计算成本。                 \n",
    "      - 添加不影响性能的参数不会降低效率。                           "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们使用SVR的例子，结合管道来进行调优："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:19:45.684778Z",
     "start_time": "2021-03-14T12:19:45.444778Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CV accuracy: 0.187 +/- 0.649\n"
     ]
    }
   ],
   "source": [
    "# 我们先来对未调参的SVR进行评价： \n",
    "from sklearn.svm import SVR     # 引入SVR类\n",
    "from sklearn.pipeline import make_pipeline   # 引入管道简化学习流程\n",
    "from sklearn.preprocessing import StandardScaler # 由于SVR基于距离计算，引入对数据进行标准化的类\n",
    "from sklearn.model_selection import GridSearchCV  # 引入网格搜索调优\n",
    "from sklearn.model_selection import cross_val_score # 引入K折交叉验证\n",
    "from sklearn import datasets\n",
    "\n",
    "\n",
    "boston = datasets.load_boston()     # 返回一个类似于字典的类\n",
    "X = boston.data\n",
    "y = boston.target\n",
    "features = boston.feature_names\n",
    "pipe_SVR = make_pipeline(StandardScaler(),\n",
    "                         SVR())\n",
    "score1 = cross_val_score(estimator=pipe_SVR,\n",
    "                         X = X,\n",
    "                         y = y,\n",
    "                         scoring = 'r2',\n",
    "                          cv = 10)       # 10折交叉验证\n",
    "print(\"CV accuracy: %.3f +/- %.3f\" % ((np.mean(score1)),np.std(score1)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:21:38.043502Z",
     "start_time": "2021-03-14T12:19:54.033501Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "网格搜索最优得分： 0.608130307081714\n",
      "网格搜索最优参数组合：\n",
      " {'svr__C': 1000.0, 'svr__gamma': 0.001, 'svr__kernel': 'rbf'}\n"
     ]
    }
   ],
   "source": [
    "# 下面我们使用网格搜索来对SVR调参：\n",
    "from sklearn.pipeline import Pipeline\n",
    "pipe_svr = Pipeline([(\"StandardScaler\",StandardScaler()),\n",
    "                     (\"svr\",SVR())])\n",
    "param_range = [0.0001,0.001,0.01,0.1,1.0,10.0,100.0,1000.0]\n",
    "param_grid = [{\"svr__C\":param_range,\"svr__kernel\":[\"linear\"]},  # 注意__是指两个下划线，一个下划线会报错的\n",
    "                {\"svr__C\":param_range,\"svr__gamma\":param_range,\"svr__kernel\":[\"rbf\"]}]\n",
    "gs = GridSearchCV(estimator=pipe_svr,\n",
    "                     param_grid = param_grid,\n",
    "                     scoring = 'r2',\n",
    "                      cv = 10)       # 10折交叉验证\n",
    "gs = gs.fit(X,y)\n",
    "print(\"网格搜索最优得分：\",gs.best_score_)\n",
    "print(\"网格搜索最优参数组合：\\n\",gs.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:21:41.823502Z",
     "start_time": "2021-03-14T12:21:38.045502Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "随机搜索最优得分： 0.2990248406082611\n",
      "随机搜索最优参数组合：\n",
      " {'svr__C': 1.6302455616216407, 'svr__gamma': 0.05695647473831755, 'svr__kernel': 'linear'}\n"
     ]
    }
   ],
   "source": [
    "# 下面我们使用随机搜索来对SVR调参：\n",
    "from sklearn.model_selection import RandomizedSearchCV\n",
    "from scipy.stats import uniform  # 引入均匀分布设置参数\n",
    "pipe_svr = Pipeline([(\"StandardScaler\",StandardScaler()),\n",
    "                     (\"svr\",SVR())])\n",
    "\n",
    "distributions = dict(svr__C=uniform(loc=1.0, scale=4),    # 构建连续参数的分布\n",
    "                     svr__kernel=[\"linear\",\"rbf\"],        # 离散参数的集合\n",
    "                    svr__gamma=uniform(loc=0, scale=4))\n",
    "\n",
    "rs = RandomizedSearchCV(estimator=pipe_svr,\n",
    "                         param_distributions = distributions,\n",
    "                         scoring = 'r2',\n",
    "                          cv = 10)       # 10折交叉验证\n",
    "rs = rs.fit(X,y)\n",
    "print(\"随机搜索最优得分：\",rs.best_score_)\n",
    "print(\"随机搜索最优参数组合：\\n\",rs.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "经过我们不懈的努力，从收集数据集并选择合适的特征、选择度量模型性能的指标、选择具体的模型并进行训练以优化模型到评估模型的性能并调参，我们认识到了如何使用sklearn构建简单回归模型。在本章的最后，我们会给出一个具体的案例，整合回归的内容。下面我们来看看机器学习另外一类大问题：分类。与回归一样，分类问题在机器学习的地位非常重要，甚至有的地方用的比回归问题还要多，因此分类问题是十分重要的！"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 使用sklearn构建完整的分类项目                   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(1) 收集数据集并选择合适的特征：在数据集上我们使用我们比较熟悉的IRIS鸢尾花数据集。\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:35:18.397590Z",
     "start_time": "2021-03-14T12:35:18.379590Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)  \\\n0                5.1               3.5                1.4               0.2   \n1                4.9               3.0                1.4               0.2   \n2                4.7               3.2                1.3               0.2   \n3                4.6               3.1                1.5               0.2   \n4                5.0               3.6                1.4               0.2   \n\n   target  \n0       0  \n1       0  \n2       0  \n3       0  \n4       0  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>sepal length (cm)</th>\n      <th>sepal width (cm)</th>\n      <th>petal length (cm)</th>\n      <th>petal width (cm)</th>\n      <th>target</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>5.1</td>\n      <td>3.5</td>\n      <td>1.4</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>4.9</td>\n      <td>3.0</td>\n      <td>1.4</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>4.7</td>\n      <td>3.2</td>\n      <td>1.3</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4.6</td>\n      <td>3.1</td>\n      <td>1.5</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>5.0</td>\n      <td>3.6</td>\n      <td>1.4</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()\n",
    "X = iris.data\n",
    "y = iris.target\n",
    "feature = iris.feature_names\n",
    "data = pd.DataFrame(X,columns=feature)\n",
    "data['target'] = y\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "各个特征的相关解释：\n",
    "   - sepal length (cm)：花萼长度(厘米)\n",
    "   - sepal width (cm)：花萼宽度(厘米)\n",
    "   - petal length (cm)：花瓣长度(厘米)\n",
    "   - petal width (cm)：花瓣宽度(厘米)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(2) 选择度量模型性能的指标：                                    \n",
    "度量分类模型的指标和回归的指标有很大的差异，首先是因为分类问题本身的因变量是离散变量，因此像定义回归的指标那样，单单衡量预测值和因变量的相似度可能行不通。其次，在分类任务中，我们对于每个类别犯错的代价不尽相同，例如：我们将癌症患者错误预测为无癌症和无癌症患者错误预测为癌症患者，在医院和个人的代价都是不同的，前者会使得患者无法得到及时的救治而耽搁了最佳治疗时间甚至付出生命的代价，而后者只需要在后续的治疗过程中继续取证就好了，因此我们很不希望出现前者，当我们发生了前者这样的错误的时候会认为建立的模型是很差的。为了解决这些问题，我们必须将各种情况分开讨论，然后给出评价指标。             \n",
    "   - 真阳性TP：预测值和真实值都为正例；                        \n",
    "   - 真阴性TN：预测值与真实值都为正例；                     \n",
    "   - 假阳性FP：预测值为正，实际值为负；\n",
    "   - 假阴性FN：预测值为负，实际值为正；                      \n",
    "   ![jupyter](./1.22.png)                                       \n",
    "分类模型的指标：                    \n",
    "   - 准确率：分类正确的样本数占总样本的比例，即：$ACC = \\frac{TP+TN}{FP+FN+TP+TN}$.                                \n",
    "   - 精度：预测为正且分类正确的样本占预测值为正的比例，即：$PRE = \\frac{TP}{TP+FP}$.                     \n",
    "   - 召回率：预测为正且分类正确的样本占类别为正的比例，即：$REC =  \\frac{TP}{TP+FN}$.                     \n",
    "   - F1值：综合衡量精度和召回率，即：$F1 = 2\\frac{PRE\\times REC}{PRE + REC}$.                                     \n",
    "   - ROC曲线：以假阳率为横轴，真阳率为纵轴画出来的曲线，曲线下方面积越大越好。                                                          \n",
    "https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics                           \n",
    "![jupyter](./1.21.png)                          \n",
    "在本次小案例中，我们使用ROC曲线作为最终评价指标。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(3) 选择具体的模型并进行训练                              \n",
    "   - **逻辑回归logistic regression：**                      \n",
    "   说到分类问题与回归问题的区别，在于回归问题与分类问题需要预测的因变量不一样。在回归问题中，因变量是连续性变量，我们需要预测$E(Y|X)$是一个连续的实数，但是在分类问题中，我们往往是通过已知X的信息预测Y的类别，往往是一个离散集合中的某个元素。如：是否患癌症，图片是猫还是狗等。一个很自然的想法是能否用线性回归去处理分类问题，答案是可以但不好！先来看看线性回归处理分类问题会出现什么弊端，我们仔细来看这个线性回归的例子，${default = \\beta_0 + \\beta_1 Balance + \\beta_2 Income}$，只要输入Balance 和 Income 以及default的数据就能用最小二乘法估计出${\\beta_0,\\beta_1}$,设定预测的default>0.5就是违约反之不违约，感觉很完美的样子，但事实真的是这样吗？假设我们需要用某个人的债务(Balance)和收入(Income)去预测是否会信用卡违约(default)：       \n",
    "      - 我们假设有一个穷人Lisa,他的Balance和Income都很小，那么有可能会导致default的值为负数，那么这个负数代表什么意义呢？显然是没有任何意义的。                \n",
    "      ![jupyter](./1.23.png)                            \n",
    "      - 当我们的分类变量是多类的时候，以0.5为界限划分分类就不可用了，那么我们应该怎么找到一个界限衡量多分类呢？                              \n",
    "   基于以上问题，现在大家是否还觉得线性回归模型作为一个分类模型是否足够优秀呢？其实，为了解决以上的问题（1）我们来想想能不能将线性回归的结果default转化为区间[0:1]上，让default转变成一个违约的概率呢？下面我们来解决这个问题吧。                              \n",
    "   在推导逻辑回归之前，我们先来认识下一组函数，这组函数具有神奇的作用，可以将是实数轴上的数转换为[0:1]区间上的概率。\n",
    "  首先，我们假设我们的线性回归模型为 ${Y=\\beta_0+\\beta_1 X}$，那么这个函数是如何将线性回归的结果转化为概率呢？这个函数就是logistic 函数，具体的形式为   ${p(X) = \\dfrac{e^{\\beta_0 + \\beta_1X}}{1+e^{\\beta_0 + \\beta_1X}}}$，他的函数图像如下图：（左边是线性回归，右边是逻辑函数）                             \n",
    "  ![jupyter](./1.24.png)                                   \n",
    "  因此，我们假设逻辑回归模型为：$p(y = 1|x) = \\frac{1}{1+e^{-w^Tx}}$ .                              \n",
    "  下面我们来具体推导下逻辑回归模型：                          \n",
    "  假设数据Data$\\{(x_i,y_i) \\},\\;\\;i = 1,2,...,N,\\;\\;x_i \\in R^p,y_i \\in \\{0,1 \\}$，设$p_1 = p(y=1|x) = \\sigma(w^T) = \\frac{1}{1+e^{-w^Tx}}$。因为y只可能取0或者1，因此假设数据服从0-1分布，也叫伯努力分布，即：当y=1时，$p(y|x)=p_1$，当y=0时，$p(y|x)=1-p_1$，可以写成$p(y|x) = p_1^y(1-p_1)^{1-y}$，可以带入y=0和y=1进去验证，结果和前面的结论一模一样。                    \n",
    "  我们使用极大似然估计MLE，即：                         \n",
    "  $$\n",
    "  \\hat{w} = argmax_w\\;\\;log\\;P(Y|X) = argmax_x\\;\\;log\\;\\prod_{i=1}^N P(y_i|x_i) = argmax_w \\sum\\limits_{i=1}^{N} log\\;P(y_i|x_i)\\\\\n",
    "  \\;\\;\\; = argmax_w \\sum\\limits_{i=1}^{N}(y_ilog\\;p_1 + (1-y_i)log(1-p_1)) \\\\ \n",
    "  记：L(w) = \\sum\\limits_{i=1}^{N}(y_ilog\\;p_1 + (1-y_i)log(1-p_1))\\\\\n",
    " \\;\\;\\; \\frac{\\partial L}{\\partial w_k} = \\sum\\limits_{i=1}^{N} y_i\\frac{1}{p_1}\\frac{\\partial p_1}{\\partial z}\\frac{\\partial z}{\\partial w_k} + (1-y_i)\\frac{1}{1-p_1}(-\\frac{\\partial p_1}{\\partial z}\\frac{\\partial z}{\\partial w_k})\\\\\n",
    "  \\;\\;\\;=\\sum\\limits_{i=1}^{N}y_i\\frac{1}{\\sigma(z)}(\\sigma(z_i)-\\sigma(z_i)^2)x_i + (1-y_i)\\frac{1}{1-\\sigma(z_i)}[-(\\sigma(z_i)-\\sigma(z_i)^2)x_i]\\\\\n",
    "  \\;\\;\\; =\\sum\\limits_{i=1}^{N}[(y_i-y_i\\sigma(z_i))x_i + (1-y_i)(-\\sigma(z_i))x_i]\\\\\n",
    "  \\;\\;\\; = \\sum\\limits_{i=1}^{N}y_ix_i-\\sigma(z_i)x_i = \\sum\\limits_{i=1}^{N}(y_i-\\sigma(z_i))x_i\n",
    "  $$                 \n",
    "  因此，$\\frac{\\partial L}{\\partial w_k} = \\sum\\limits_{i=1}^{N}(y_i-\\sigma(z_i))x_i$，由于这里涉及的函数不像线性回归一样能简单求出解析解，因此我们使用迭代的优化算法：梯度下降法，即：                       \n",
    "  $w_k^{(t+1)}\\leftarrow w_k^{(t)} - \\eta \\sum\\limits_{i=1}^{N}(y_i-\\sigma(z_i))x_i^{(k)},\\;\\;\\;其中，x_i^{(k)}为第i个样本第k个特征$                                 \n",
    "  如果想了解关于梯度下降法等无约束算法的具体细节，可以参照笔者写的另外两篇知乎博客：                          \n",
    "  最优化理论之无约束优化基本结构及其python应用：https://zhuanlan.zhihu.com/p/163405865                                                   \n",
    "  最优化理论之负梯度方法与Newton型方法：https://zhuanlan.zhihu.com/p/165914126                                              \n",
    "  对于问题(2),我们值得注意的是，逻辑回归在实际中不太用于多分类问题，因为实际效果不是很好，所以我们可以借助其他模型来解决这个问题，那让我们来解决这个遗留下来的问题吧。                                \n",
    " \n",
    "             "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 基于概率的分类模型：                               \n",
    "   (1) 线性判别分析：                                              \n",
    "   线性判别分析是一个比较久远的算法，我将会从两个方向去描述这个算法，因为我觉得每位读者都有自己喜欢的那个理解的方向，分别是基于贝叶斯公式和降维分类的思想。                        \n",
    "      - 基于贝叶斯公式对线性判别分析的理解：                       \n",
    "   在讨论如何解决多分类问题之前，我们先来说说贝叶斯的那些事吧。在概率统计的领域里有一条神奇的公式叫贝叶斯定理，具体的形式是：${P(Y=k|X=x) = \\dfrac{{\\pi}_kf_k(x)}{\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)}}$ ，我们 先不要被公式的符号吓到，我们先来看看符号具体代表什么意思。我们假设观测有${K}$类，${\\pi_k}$为随机选择的观测来自第${k}$类的 __先验概率__，也就是样本里面第${k}$类的样本个数除以总样本的个数：${\\pi_k = \\dfrac{n_k}{n}}$。再来 ${f_k(x) =P(X=x|Y=k)}$，表示第${k}$类观测的X的密度函数，说的直白一点就是在${Y=k}$的样本里${X=x}$的样本个数，即${f_k(x) = P(X=x|Y=k) = \\dfrac{n_{(X=x,Y=k)}}{n_{(Y=k)}}}$，最后，${\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)}=P(X=x)=\\dfrac{n_{(X=x)}}{n}$，也就是样本中${X=x}$的概率。\n",
    "      在讨论贝叶斯定理后，我们回到分类问题，这个定理跟我们的分类问题有什么关联呢？没错，这个公式${P(Y=k|X=x) = \\dfrac{{\\pi}_kf_k(x)}{\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)}}$给出了给定样本条件下，${Y=k}$这个类别下的概率，这给分类问题提供了一条思路，那就是计算这个${P(Y=k|X=x)}$，而且我们的逻辑回归就是这么干的，但是在${P(Y=k|X=x) = \\dfrac{{\\pi}_kf_k(x)}{\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)}}$这个公式中，分母${{\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)} = P(X=x)}$当样本给定的时候是一个与分类${k}$无关的常数,所以我们的问题可以简化为只需要计算分子${{\\pi}_kf_k(x)}$,进而比较哪个类别的概率最大就知道属于哪个类别了，因此我们的分类思路就出来啦，这个思路不同于逻辑回归，逻辑回归需要计算具体的${P(Y=k|X=x)}$概率值，而我们现在的思路是通过贝叶斯定理计算贝叶斯定理的分子，比较分子最大的那个类别为最终类别。                 \n",
    "      在我们推导复杂算法之前，我们先推导下简单的当自变量个数只有一个的模型，即${p=1}$的简单模型。我们记${P(Y=k|X=x) = \\dfrac{{\\pi}_kf_k(x)}{\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)}}$ 的分子为${g_k(x) = {\\pi}_kf_k(x)}$。在这里，我们做个模型假设：假设${f_k(x) }$服从正态分布，即${f_k(x) \\sim N(\\mu,\\sigma_k^2)}$，而且每个${\\sigma_k^2 = \\sigma^2}$，同方差假设。因此${f_k(x) = \\dfrac{1}{\\sqrt{2\\pi}\\sigma_k}e^{-\\dfrac{1}{2\\sigma^2}(x-\\mu_k)^2}}$，最终我们的${g_k(x) = \\pi_k\\dfrac{1}{\\sqrt{2\\pi}\\sigma_k}e^{-\\dfrac{1}{2\\sigma^2}(x-\\mu_k)^2}}$,终于算出来啦。这个式子不是很好计算，我们对${g_k(x)}$取个对数，令${\\delta_k(x) = ln(g_k(x))=ln\\pi_k+\\dfrac{\\mu}{\\sigma^2}x-\\dfrac{\\mu^2}{2\\sigma^2}}$，到这里我们的模型建立模型，我们只需要把未知的${\\mu_k}$与${\\sigma^2}$估计出来就好了。${\\hat{\\mu}_k =\\dfrac{1}{n_k}\\sum\\limits_{i:y_i=k}x_i}$，也就是当${y=k}$这一类中${x}$的平均值；${\\hat{\\sigma}^2 =\\dfrac{1}{n-K}\\sum\\limits_{k=1}^K\\sum\\limits_{i:y_i=k}(x_i-\\hat{\\mu}_k)^2 }$，说白了就是计算每一类的方差，再求平均值。总结下上面的公式就是：                                    \n",
    "${\\begin{cases}\\delta_k(x) = ln(g_k(x))=ln\\pi_k+\\dfrac{\\mu}{\\sigma^2}x-\\dfrac{\\mu^2}{2\\sigma^2}\\\\{\\hat{\\mu}_k =\\dfrac{1}{n_k}\\sum\\limits_{i:y_i=k}x_i}\\\\{\\hat{\\sigma}^2 =\\dfrac{1}{n-K}\\sum\\limits_{k=1}^K\\sum\\limits_{i:y_i=k}(x_i-\\hat{\\mu}_k)^2}\\end{cases}}$                              \n",
    "      至此，我们的模型就建立完成了，我们只需要代入数据求出${\\delta_k(x)}$，哪个${k}$对应的${\\delta_k(x)}$大，就是哪一类。                                   \n",
    "   （下图虚线是线性判别分析的决策边界，正态曲线哪边高样本就是哪一类）                  \n",
    "      ![jupyter](./1.25.png)                            \n",
    "      我们推到出了一个自变量的简单模型，就要泛化为多个自变量的线性判别分析了，即${p>1}$。其实原理一样的，只是将一元正态分布扩展为多元正态分布：\n",
    "      ${f_k(x)=\\dfrac{1}{(2\\pi)^{\\tfrac{p}{2}}|\\Sigma|^\\tfrac{1}{2}}e^{[-\\tfrac{1}{2}(x-\\mu_k)^T\\Sigma^{-1}(x-\\mu_k)]}}$                           \n",
    "      ${\\hat{\\mu_k}=(\\mu_{k1},\\mu_{k2},......,\\mu_{kp})   ,   \\hat{\\Sigma}=\\dfrac{1}{p-1}\\sum\\limits_{j=1}^p(x_j-\\overline{x})(x_j-\\overline{x})^T}$                               \n",
    "      ${\\delta_k(x) = ln(\\pi_kf_k(x))=ln(\\pi_k)-(\\dfrac{p}{2}ln(2\\pi)+\\dfrac{1}{2}ln(|\\Sigma|))-\\dfrac{1}{2}(x-\\mu_k)^T\\Sigma^-1(x-\\mu_k)=x^T\\hat{\\Sigma}\\hat{\\mu}_k-\\dfrac{1}                                                       {2}\\hat{\\mu}_k^T\\hat{\\Sigma}^{-1}\\hat{\\mu}_k+ln\\hat{\\pi}_k}$                            \n",
    "      - 降维分类的思想理解线性判别分析：                   \n",
    "      基于数据进行分类时，一个很自然的想法是：将高维的数据降维至一维，然后使用某个阈值将各个类别分开。下面用图的形式展示：                   \n",
    "      ![jupyter](./1.26.png)                        \n",
    "      图中，数据的维度是二维的，我们的想法是把数据降维至一维，然后用阈值就能分类。这个似乎是一个很好的想法，我们总是希望降维后的数据同一个类别自身内部方差小，不同类别之间的方差要尽可能大。这也是合理的，因为同一个类别的数据应该更加相似，因此方差小；不同类别的数据之间应该很不相似，这样才能更容易对数据进行分类，我们简称为：**类内方差小，类间方差大**，在计算机语言叫“松耦合，高内聚”。在做具体的推导之前，我们对数据的形式和一些基本统计量做一些描述：                            \n",
    "      特征$X = (x_1,x_2,...,x_N)^T$，因变量$Y = (y_1,y_2,...,y_N)^T,\\;\\;其中，y_i \\in \\{+1,-1 \\}$，类别c1的特征$X_{c_1} = \\{x_i|y_i=+1 \\}$，同理，类别c2的特征$X_{c_2} = \\{x_i|y_i=-1 \\}$，属于c1类别的数据个数为$N_1$，属于类别c2的数据个数为$N_2$，其中，$N_1+N_2 = N$。                         \n",
    "      特征X投影在w方向至一维：$z_i = w^Tx_i,\\;\\;||w|| = 1$                            \n",
    "      全样本投影的均值$\\bar{z} = \\frac{1}{N}\\sum\\limits_{i=1}^{N}z_i = \\frac{1}{N}\\sum\\limits_{i=1}^{N}w^Tx_i$                    \n",
    "      全样本投影的协方差$S_z = \\frac{1}{N}\\sum\\limits_{i=1}^{N}(z_i-\\bar{z})(z_i-\\bar{z})^T = \\frac{1}{N}\\sum\\limits_{i=1}^{N}(w^Tx_i-\\bar{z})(w^Tx_i-\\bar{z})^T$                   \n",
    "      c1样本投影的均值$\\bar{z_1} = \\frac{1}{N_1}\\sum\\limits_{i=1}^{N_1}z_i = \\frac{1}{N_1}\\sum\\limits_{i=1}^{N_1}w^Tx_i$                    \n",
    "      c1样本投影的协方差$S_{z_1} = \\frac{1}{N_1}\\sum\\limits_{i=1}^{N_1}(z_i-\\bar{z_1})(z_i-\\bar{z_1})^T = \\frac{1}{N_1}\\sum\\limits_{i=1}^{N_1}(w^Tx_i-\\bar{z_1})(w^Tx_i-\\bar{z_1})^T$                       \n",
    "      c2样本投影的均值 $\\bar{z_2} = \\frac{1}{N_2}\\sum\\limits_{i=1}^{N_2}z_i = \\frac{1}{N_2}\\sum\\limits_{i=1}^{N_2}w^Tx_i$                     \n",
    "      c2样本投影的协方差$S_{z_2} = \\frac{1}{N_2}\\sum\\limits_{i=1}^{N_2}(z_i-\\bar{z_2})(z_i-\\bar{z_2})^T = \\frac{1}{N_2}\\sum\\limits_{i=1}^{N_2}(w^Tx_i-\\bar{z_2})(w^Tx_i-\\bar{z_2})^T$                      \n",
    "      类间差距：$(\\bar{z}_1-\\bar{z}_2)^2$                      \n",
    "      类内方差：$S_1 + S_2$                          \n",
    "      由于线性判别分析的目标是同一类别内方差小，不同类别之间距离大，因此损失函数定义为：   \n",
    "                            \n",
    "   $$\n",
    "      J(w) = \\frac{(\\bar{z}_1-\\bar{z}_2)^2}{s_1+s_2} = \\frac{w^T(\\bar{x}_{c_1}-\\bar{x}_{c_2})(\\bar{x}_{c_1}-\\bar{x}_{c_2})^Tw}{w^T(s_{c_1}+s_{c_2})w}\\\\\n",
    "      \\;\\;\\; \\hat{w} = argmax_w\\;J(w)\n",
    "   $$                             \n",
    "   记：$S_b = (\\bar{x}_{c_1}-\\bar{x}_{c_2})(\\bar{x}_{c_1}-\\bar{x}_{c_2})^T,\\;S_w = (s_{c_1}+s_{c_2})$，因此$J(w) = \\frac{w^TS_bw}{w^TS_ww}$                   \n",
    "   让J(w)对w求导等于0，求出：$w = S_w^{-1}(\\bar{x}_{c_1}-\\bar{x}_{c_2})$                       \n",
    "   (2) 朴素贝叶斯：                                        \n",
    "   在线性判别分析中，我们假设每种分类类别下的特征遵循同一个协方差矩阵，每两个特征之间是存在协方差的，因此在线性判别分析中各种特征是不是独立的。但是，朴素贝叶斯算法对线性判别分析作进一步的模型简化，它将线性判别分析中的协方差矩阵中的协方差全部变成0，只保留各自特征的方差，也就是朴素贝叶斯假设各个特征之间是不相关的。在之前所看到的偏差-方差理论中，我们知道模型的简化可以带来方差的减少但是增加偏差，因此朴素贝叶斯也不例外，它比线性判别分析模型的方差小，偏差大。虽然简化了模型，实际中使用朴素贝叶斯的案例非常多，甚至多于线性判别分析，例如鼎鼎大名的新闻分类，垃圾邮件分类等。\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:35:23.668587Z",
     "start_time": "2021-03-14T12:35:23.637593Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "e:\\a_projects\\pyprojects\\py_38_project\\venv\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:763: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  n_iter_i = _check_optimize_result(\n"
     ]
    },
    {
     "data": {
      "text/plain": "0.9733333333333334"
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  逻辑回归\n",
    "'''\n",
    "penalty       {‘l1’, ‘l2’, ‘elasticnet’, ‘none’}, default=’l2’正则化方式\n",
    "dual      bool, default=False   是否使用对偶形式，当n_samples> n_features时，默认dual = False。   \n",
    "C        float, default=1.0      \n",
    "solver       {‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’, ‘saga’}, default=’lbfgs’     \n",
    "l1_ratio         float, default=None           \n",
    "'''\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "log_iris = LogisticRegression()\n",
    "log_iris.fit(X,y)\n",
    "log_iris.score(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:36:00.016035Z",
     "start_time": "2021-03-14T12:35:59.990036Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "0.98"
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 线性判别分析\n",
    "'''\n",
    "参数：\n",
    "solver:{'svd'，'lsqr'，'eigen'}，默认='svd'\n",
    "solver的使用，可能的值：\n",
    "'svd'：奇异值分解（默认）。不计算协方差矩阵，因此建议将此求解器用于具有大量特征的数据。\n",
    "\n",
    "'lsqr'：最小二乘解，可以与收缩结合使用。\n",
    "\n",
    "'eigen'：特征值分解，可以与收缩结合使用。\n",
    "'''\n",
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis\n",
    "lda_iris = LinearDiscriminantAnalysis()\n",
    "lda_iris.fit(X,y)\n",
    "lda_iris.score(X,y)\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:36:06.451598Z",
     "start_time": "2021-03-14T12:36:06.437598Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "0.96"
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 朴素贝叶斯             \n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "NB_iris = GaussianNB()\n",
    "NB_iris.fit(X, y)\n",
    "NB_iris.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 决策树 ：                     \n",
    "   与前面内容所讲的决策树回归大致是一样的，只是在回归问题中，选择分割点的标准是均方误差，但是在分类问题中，由于因变量是类别变量而不是连续变量，因此用均方误差显然不合适。那问题是用什么作为选择分割点的标准呢？我们先来分析具体的问题：                         \n",
    "   在回归树中，对一个给定的观测值，因变量的预测值取它所属的终端结点内训练集的平均因变量。与之相对应，对于分类树来说，给定一个观测值，因变量的预测值为它所属的终端结点内训练集的**最常出现的类**。分类树的构造过程与回归树也很类似，与回归树一样，分类树也是采用递归二叉分裂。但是在分类树中，均方误差无法作为确定分裂节点的准则，一个很自然的替代指标是分类错误率。分类错误率就是：此区域内的训练集中非常见类所占的类别，即：                                   \n",
    "   $$\n",
    "   E = 1-max_k(\\hat{p}_{mk})\n",
    "   $$                       \n",
    "   上式中的$\\hat{p}_{mk}$代表第m个区域的训练集中第k类所占的比例。但是在大量的事实证明：分类错误率在构建决策树时不够敏感，一般在实际中用如下两个指标代替：             \n",
    "   (1) 基尼系数：                   \n",
    "   $$\n",
    "   G = \\sum\\limits_{k=1}^{K} \\hat{p}_{mk}(1-\\hat{p}_{mk})\n",
    "   $$             \n",
    "   在基尼系数的定义中，我们发现这个指标衡量的是K个类别的总方差。不难发现，如果所有的$\\hat{p}_{mk}$的取值都接近0或者1，基尼系数会很小。因此基尼系数被视为衡量结点纯度的指标----如果他的取值小，那就意味着某个节点包含的观测值几乎来自同一个类别。                         \n",
    "   由基尼系数作为指标得到的分类树叫做：CART。                        \n",
    "   (2) 交叉熵：                       \n",
    "   可以替代基尼系数的指标是交叉熵，定义如下：                           \n",
    "   $$\n",
    "   D = -\\sum\\limits_{k=1}^{K} \\hat{p}_{mk}log\\;\\hat{p}_{mk}\n",
    "   $$                     \n",
    "   显然，如果所有的$\\hat{p}_{mk}$都接近于0或者1，那么交叉熵就会接近0。因此，和基尼系数一样，如果第m个结点的纯度越高，则交叉熵越小。事实证明，基尼系数和交叉熵在数值上时很接近的。                   \n",
    "   \n",
    "   ![jupyter](./1.27.png)                                            \n",
    "   决策树分类算法的完整步骤：                          \n",
    "      a.  选择最优切分特征j以及该特征上的最优点s：                \n",
    "      遍历特征j以及固定j后遍历切分点s，选择使得基尼系数或者交叉熵最小的(j,s)                                                   \n",
    "       b. 按照(j,s)分裂特征空间，每个区域内的类别为该区域内样本比例最多的类别。                           \n",
    "       c. 继续调用步骤1，2直到满足停止条件，就是每个区域的样本数小于等于5。        \n",
    "       d. 将特征空间划分为J个不同的区域，生成分类树。                 \n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:40:47.337041Z",
     "start_time": "2021-03-14T12:40:47.324041Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "0.9733333333333334"
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 使用决策树算法对iris分类：\n",
    "'''\n",
    "criterion:{“gini”, “entropy”}, default=”gini”\n",
    "max_depth:树的最大深度。\n",
    "min_samples_split:拆分内部节点所需的最少样本数\n",
    "min_samples_leaf :在叶节点处需要的最小样本数。\n",
    "\n",
    "'''\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "tree_iris = DecisionTreeClassifier(min_samples_leaf=5)\n",
    "tree_iris.fit(X,y)\n",
    "tree_iris.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 支持向量机SVM：                                   \n",
    "   支持向量机SVM是20世纪90年代在计算机界发展起来的一种分类算法，在许多问题中都被证明有较好的效果，被认为是适应性最广的算法之一。                         \n",
    "   ![jupyter](./1.28.png)                               \n",
    "   支持向量机的基本原理非常简单，如图所视，白色和蓝色的点各为一类，我们的目标是找到一个分割平面将两个类别分开。通常来说，如果数据本身是线性可分的，那么事实上存在无数个这样的超平面。这是因为给定一个分割平面稍微上移下移或旋转这个超平面，只要不接触这些观测点，仍然可以将数据分开。一个很自然的想法就是找到**最大间隔超平面**，即找到一个分割平面距离最近的观测点最远。下面我们来严格推导：                   \n",
    "   我们根据距离超平米那最近的点，只要同时缩放w和b可以得到：$w^Tx_1 + b = 1$与$w^Tx_2+b = -1$，因此：                      \n",
    "   $$\n",
    "  \\begin{array}{l}\n",
    "   w^{T} x_{1}+b=1 \\\\\n",
    "    w^{T} x_{2}+b=-1 \\\\\n",
    "    \\left(w^{T} x_{1}+b\\right)-\\left(w^{T} x_{2}+b\\right)=2 \\\\\n",
    "   w^{T}\\left(x_{1}-x_{2}\\right)=2 \\\\\n",
    "   \\qquad \\begin{array}{l}\n",
    "   w^{T}\\left(x_{1}-x_{2}\\right)=\\|w\\|_{2}\\left\\|x_{1}-x_{2}\\right\\|_{2} \\cos \\theta=2 \\\\\n",
    "   \\left\\|x_{1}-x_{2}\\right\\|_{2} \\cos \\theta=\\frac{2}{\\|w\\|_{2}}\n",
    "   \\end{array} \\\\\n",
    "    \\qquad \\begin{array}{l}\n",
    "   d_{1}=d_{2}=\\frac{\\left\\|x_{1}-x_{2}\\right\\|_{2} \\cos \\theta}{2}=\\frac{\\frac{2}{\\|w\\|_{2}}}{2}=\\frac{1}{\\|w\\|_{2}} \\\\\n",
    "   d_{1}+d_{2}=\\frac{2}{\\|w\\|_{2}}\n",
    "   \\end{array}\n",
    "   \\end{array}\n",
    "   $$                        \n",
    "   由此可知道SVM模型的具体形式：                           \n",
    "   $$\n",
    "  \\begin{aligned}\n",
    "\\min _{w, b} & \\frac{1}{2}\\|w\\|^{2} \\\\\n",
    "\\text { s.t. } & y^{(i)}\\left(w^{T} x^{(i)}+b\\right) \\geq 1, \\quad i=1, \\ldots, n\n",
    "\\end{aligned}\n",
    "   $$                     \n",
    "   可以将约束条件写为: $g_{i}(w)=-y^{(i)}\\left(w^{T} x^{(i)}+b\\right)+1 \\leq 0  $                                                     \n",
    "   可以将优化问题拉格朗日化\n",
    "   $$\n",
    "   \\mathcal{L}(w, b, \\alpha)=\\frac{1}{2}\\|w\\|^{2}-\\sum_{i=1}^{n} \\alpha_{i}\\left[y^{(i)}\\left(w^{T} x^{(i)}+b\\right)-1\\right]\n",
    "   $$                        \n",
    "   因此：                   \n",
    "   $$\n",
    "   \\mathcal{L}(w, b, \\alpha)=\\frac{1}{2}\\|w\\|^{2}-\\sum_{i=1}^{n} \\alpha_{i}\\left[y^{(i)}\\left(w^{T} x^{(i)}+b\\right)-1\\right]\n",
    "   $$                   \n",
    "   欲构造 dual 问题, 首先求拉格朗日化的问题中  $\\mathrm{w} $ 和  $\\mathrm{b} $ 的值, 对 $ \\mathrm{w}$  求梯度, 令梯度为  0,  可求得 w:              \n",
    "   对 b 求梯度, 令梯度为 0, 可得：              \n",
    "   $$\n",
    "   \\frac{\\partial}{\\partial b} \\mathcal{L}(w, b, \\alpha)=\\sum_{i=1}^{n} \\alpha_{i} y^{(i)}=0\n",
    "   $$               \n",
    "\n",
    "   将  $\\mathrm{w}$  带入拉格朗日化的原问题可得                        \n",
    "   $$\n",
    "   \\begin{array}{l}\n",
    "   \\mathcal{L}(w, b, \\alpha)=\\sum_{i=1}^{n} \\alpha_{i}-\\frac{1}{2} \\sum_{i, j=1}^{n} y^{(i)} y^{(j)} \\alpha_{i} \\alpha_{j}\\left(x^{(i)}\\right)^{T} x^{(j)}-b \\sum_{i=1}^{n} \\alpha_{i} y^{(i)} \\\\\n",
    "   \\mathcal{L}(w, b, \\alpha)=\\sum_{i=1}^{n} \\alpha_{i}-\\frac{1}{2} \\sum_{i, j=1}^{n} y^{(i)} y^{(j)} \\alpha_{i} \\alpha_{j}\\left(x^{(i)}\\right)^{T} x^{(j)}\n",
    "   \\end{array}\n",
    "   $$                           \n",
    "   因此：                           \n",
    "   $$\n",
    "   \\begin{aligned}\n",
    "    &\\text { 对拉格朗日化的原问题求最小值, 得到了 } \\mathrm{w} \\text { , 现在可以构造 dual 问題 }\\\\\n",
    "    &\\begin{aligned}\n",
    "    \\max _{\\alpha} & W(\\alpha)=\\sum_{i=1}^{n} \\alpha_{i}-\\frac{1}{2} \\sum_{i, j=1}^{n} y^{(i)} y^{(j)} \\alpha_{i} \\alpha_{j}\\left\\langle x^{(i)}, x^{(j)}\\right\\rangle \\\\\n",
    "    \\text { s.t. } & \\alpha_{i} \\geq 0, \\quad i=1, \\ldots, n \\\\\n",
    "    & \\sum_{i=1}^{n} \\alpha_{i} y^{(i)}=0\n",
    "    \\end{aligned}\\\\\n",
    "    &\\text { 可以推导出 b的值为: } b^{*}=-\\frac{\\max _{i: y^{(i)}=-1} w^{* T} x^{(i)}+\\min _{i: y^{(i)}=1} w^{* T} x^{(i)}}{2}\\\\\n",
    "    &\\begin{array}{r}\n",
    "    \\text { SVM的决策子如下,值的符号为类别. } \\\\\n",
    "    \\qquad w^{T} x+b=\\left(\\sum_{i=1}^{n} \\alpha_{i} y^{(i)} x^{(i)}\\right)^{T} x+b=\\sum_{i=1}^{n} \\alpha_{i} y^{(i)}\\left\\langle x^{(i)}, x\\right\\rangle+b\n",
    "    \\end{array}\n",
    "    \\end{aligned}\n",
    "    $$                              \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 非线性支持向量机：                     \n",
    "   在刚刚的讨论中，我们都是着重讨论了线性支持向量机是如何工作的，但是在现实生活中，我们很难碰到线性可分的数据集，如：             \n",
    "   ![jupyter](./1.29.png)                         \n",
    "   那我们应该如何处理非线性问题呢？答案就是将数据投影至更加高的维度！                    \n",
    "   ![jupyter](./1.30.png)                     \n",
    "   上图中，在一维数据做不到线性可分，我们将数据投影至二维平面就可以成功线性可分。那么，我们来详细探讨下这其中的奥妙：                        \n",
    "   \n",
    "   $$\n",
    "   \\begin{array}{l}\n",
    "   \\Phi: \\mathcal{X} \\mapsto \\hat{\\mathcal{X}}=\\Phi(\\mathbf{x}) \\\\\n",
    "   \\Phi\\left(\\left[x_{i 1}, x_{i 2}\\right]\\right)=\\left[x_{i 1}, x_{i 2}, x_{i 1} x_{i 2}, x_{i 1}^{2}, x_{i 2}^{2}\\right]\n",
    "   \\end{array}\n",
    "   $$                      \n",
    "   如果我们使用上面公式的形式将低维数据拓展至高维数据，则必须面临一个很大的问题，那就是：维度爆炸导致的计算量太大的问题。假如是一个2维特征的数据，我们可以将其映射到5维来做特征的内积，如果原始空间是三维，可以映射到到19维空间，似乎还可以处理。但是如果我们的低维特征是100个维度，1000个维度呢？那么我们要将其映射到超级高的维度来计算特征的内积。这时候映射成的高维维度是爆炸性增长的，这个计算量实在是太大了，而且如果遇到无穷维的情况，就根本无从计算了。能不能呢个避免这个问题呢？核函数隆重登场：                       \n",
    "   回顾线性可分SVM的优化目标函数：                  \n",
    "   $$\n",
    "   \\underbrace{ min }_{\\alpha}  \\frac{1}{2}\\sum\\limits_{i=1,j=1}^{m}\\alpha_i\\alpha_jy_iy_jx_i \\bullet x_j - \\sum\\limits_{i=1}^{m}\\alpha_i\\\\\n",
    "   s.t. \\; \\sum\\limits_{i=1}^{m}\\alpha_iy_i = 0\\\\\n",
    "   0 \\leq \\alpha_i \\leq C\n",
    "   $$                  \n",
    "   注意到上式低维特征仅仅以内积$x_i \\bullet x_j$ 的形式出现，如果我们定义一个低维特征空间到高维特征空间的映射$\\phi$，将所有特征映射到一个更高的维度，让数据线性可分，我们就可以继续按前两篇的方法来优化目标函数，求出分离超平面和分类决策函数了。也就是说现在的SVM的优化目标函数变成：                         \n",
    "   $$\n",
    "   \\begin{array}{c}\n",
    "   \\underbrace{\\min }_{\\alpha} \\frac{1}{2} \\sum_{i=1, j=1}^{m} \\alpha_{i} \\alpha_{j} y_{i} y_{j} \\phi\\left(x_{i}\\right) \\bullet \\phi\\left(x_{j}\\right)-\\sum_{i=1}^{m} \\alpha_{i} \\\\\n",
    "   \\text { s. } t . \\sum_{i=1}^{m} \\alpha_{i} y_{i}=0 \\\\\n",
    "   0 \\leq \\alpha_{i} \\leq C\n",
    "   \\end{array}\n",
    "   $$                    \n",
    "   可以看到，和线性可分SVM的优化目标函数的区别仅仅是将内积$x_i \\bullet x_j$替换为$\\phi(x_i) \\bullet \\phi(x_j)$。我们要将其映射到超级高的维度来计算特征的内积。这时候映射成的高维维度是爆炸性增长的，这个计算量实在是太大了，而且如果遇到无穷维的情况，就根本无从计算了。下面引入核函数：               \n",
    "   假设$\\phi$是一个从低维的输入空间$\\chi$（欧式空间的子集或者离散集合）到高维的希尔伯特空间的$\\mathcal{H}$映射。那么如果存在函数$K(x,z)$，对于任意$x, z \\in \\chi$，都有：                    \n",
    "   $$\n",
    "   K(x, z) = \\phi(x) \\bullet \\phi(z)\n",
    "   $$                       \n",
    "   那么我们就称$K(x, z)$为核函数。                   \n",
    "   仔细发现，$K(x, z)$的计算是在低维特征空间来计算的，它避免了在刚才我们提到了在高维维度空间计算内积的恐怖计算量。也就是说，我们可以好好享受在高维特征空间线性可分的利益，却避免了高维特征空间恐怖的内积计算量。下面介绍几种常用的核函数：                     \n",
    "   (1)  多项式核函数：                   \n",
    "   多项式核函数（Polynomial Kernel）是线性不可分SVM常用的核函数之一，表达式为：                         \n",
    "   $$\n",
    "   K\\left(\\mathbf{x}_{i}, \\mathbf{x}_{j}\\right)=\\left(\\left\\langle\\mathbf{x}_{i}, \\mathbf{x}_{j}\\right\\rangle+c\\right)^{d}\n",
    "   $$             \n",
    "   C用来控制低阶项的强度，C=0,d=1代表无核函数。                       \n",
    "   (2) 高斯核函数：                                                   \n",
    "   高斯核函数（Gaussian Kernel），在SVM中也称为径向基核函数（Radial Basis Function,RBF），它是非线性分类SVM最主流的核函数。libsvm默认的核函数就是它。表达式为：                             \n",
    "   $$\n",
    "   K\\left(\\mathbf{x}_{i}, \\mathbf{x}_{j}\\right)=\\exp \\left(-\\frac{\\left\\|\\mathbf{x}_{i}-\\mathbf{x}_{j}\\right\\|_{2}^{2}}{2 \\sigma^{2}}\\right)\n",
    "   $$                              \n",
    "   使用高斯核函数之前需要将特征标准化，因此这里衡量的是样本之间的相似度。                    \n",
    "   (3) Sigmoid核函数：                    \n",
    "   Sigmoid核函数（Sigmoid Kernel）也是线性不可分SVM常用的核函数之一，表达式为：               \n",
    "   $$\n",
    "   K\\left(\\mathbf{x}_{i}, \\mathbf{x}_{j}\\right)=\\tanh \\left(\\alpha \\mathbf{x}_{i}^{\\top} \\mathbf{x}_{j}+c\\right)\n",
    "   $$                        \n",
    "   此时的SVM相当于没有隐藏层的简单神经网络。                     \n",
    "   (4) 余弦相似度核：                  \n",
    "   常用于衡量两段文字的余弦相似度，表达式为：                    \n",
    "   $$\n",
    "   K\\left(\\mathbf{x}_{i}, \\mathbf{x}_{j}\\right)=\\frac{\\mathbf{x}_{i}^{\\top} \\mathbf{x}_{j}}{\\left\\|\\mathbf{x}_{i}\\right\\|\\left\\|\\mathbf{x}_{j}\\right\\|}\n",
    "   $$\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:40:54.152402Z",
     "start_time": "2021-03-14T12:40:54.139402Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "0.9733333333333334"
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.svm import SVC\n",
    "'''\n",
    "C:正则化参数。正则化的强度与C成反比。必须严格为正。惩罚是平方的l2惩罚。\n",
    "kernel:{'linear'，'poly'，'rbf'，'sigmoid'，'precomputed'}，默认='rbf'\n",
    "degree:多项式和的阶数\n",
    "gamma:“ rbf”，“ poly”和“ Sigmoid”的内核系数。\n",
    "shrinking:是否软间隔分类，默认true\n",
    "\n",
    "'''\n",
    "svc_iris = make_pipeline(StandardScaler(), SVC(gamma='auto'))\n",
    "svc_iris.fit(X, y)\n",
    "svc_iris.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(4) 评估模型的性能并调参:                        \n",
    "更详细的可以查看笔者的知乎：https://zhuanlan.zhihu.com/p/140040705"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:41:04.641260Z",
     "start_time": "2021-03-14T12:41:01.014259Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "网格搜索经历时间：3.636 S\n",
      "0.9800000000000001\n",
      "{'svc__C': 1.0, 'svc__gamma': 0.1, 'svc__kernel': 'rbf'}\n"
     ]
    }
   ],
   "source": [
    "# 使用网格搜索进行超参数调优：\n",
    "# 方式1：网格搜索GridSearchCV()\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.svm import SVC\n",
    "import time\n",
    "\n",
    "start_time = time.time()\n",
    "pipe_svc = make_pipeline(StandardScaler(),SVC(random_state=1))\n",
    "param_range = [0.0001,0.001,0.01,0.1,1.0,10.0,100.0,1000.0]\n",
    "param_grid = [{'svc__C':param_range,'svc__kernel':['linear']},{'svc__C':param_range,'svc__gamma':param_range,'svc__kernel':['rbf']}]\n",
    "gs = GridSearchCV(estimator=pipe_svc,param_grid=param_grid,scoring='accuracy',cv=10,n_jobs=-1)\n",
    "gs = gs.fit(X,y)\n",
    "end_time = time.time()\n",
    "print(\"网格搜索经历时间：%.3f S\" % float(end_time-start_time))\n",
    "print(gs.best_score_)\n",
    "print(gs.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:41:06.386333Z",
     "start_time": "2021-03-14T12:41:06.243333Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "随机网格搜索经历时间：0.097 S\n",
      "0.9733333333333334\n",
      "{'svc__kernel': 'linear', 'svc__C': 0.1}\n"
     ]
    }
   ],
   "source": [
    "# 方式2：随机网格搜索RandomizedSearchCV()\n",
    "from sklearn.model_selection import RandomizedSearchCV\n",
    "from sklearn.svm import SVC\n",
    "import time\n",
    "\n",
    "start_time = time.time()\n",
    "pipe_svc = make_pipeline(StandardScaler(),SVC(random_state=1))\n",
    "param_range = [0.0001,0.001,0.01,0.1,1.0,10.0,100.0,1000.0]\n",
    "param_grid = [{'svc__C':param_range,'svc__kernel':['linear']},{'svc__C':param_range,'svc__gamma':param_range,'svc__kernel':['rbf']}]\n",
    "# param_grid = [{'svc__C':param_range,'svc__kernel':['linear','rbf'],'svc__gamma':param_range}]\n",
    "gs = RandomizedSearchCV(estimator=pipe_svc, param_distributions=param_grid,scoring='accuracy',cv=10,n_jobs=-1)\n",
    "gs = gs.fit(X,y)\n",
    "end_time = time.time()\n",
    "print(\"随机网格搜索经历时间：%.3f S\" % float(end_time-start_time))\n",
    "print(gs.best_score_)\n",
    "print(gs.best_params_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**当类别为两类时，可以绘制混淆矩阵与ROC曲线**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:41:12.903180Z",
     "start_time": "2021-03-14T12:41:10.122598Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 180x180 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALQAAAC4CAYAAABKD8ZJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAARwklEQVR4nO3dfVRTZ54H8C8YEFkgQCIovhRrdTQqrYAs4BuRuCpia6kCM3ZcWxlahVLP8bQq7rZltK06Wt+hLkbqy3hEj7Z1RtdWXCoFfMFyXFSqNj3HASySkBcIVMDkZv9wTY3yckGSm3n4fc7xHG9yzf3m+uXyJLm5j4vFYrGAEEa4Ch2AkN5EhSZMoUITplChCVOo0IQpVGjCFJHQAZzV1atXkZeXB47jEBsbi/nz5wsdSTDZ2dkoLy+HWCzGli1bhI7TKTpCt4PjOCiVSmRmZmLr1q0oKSlBTU2N0LEEExMTg8zMTKFj8EKFbodKpcKgQYMQGBgIkUiE6OholJWVCR1LMDKZDF5eXkLH4IUK3Q6dTgeJRGJdlkgk0Ol0AiYifFGhCVOo0O3w9/eHVqu1Lmu1Wvj7+wuYiPBFhW7HyJEjUVtbC7VaDZPJhNLSUoSHhwsdi/DgQmfbta+8vBz79+8Hx3GQy+VISEgQOpJgtm3bhsrKShiNRojFYiQmJmLGjBlCx2oXFZowhYYchClUaMIUKjRhChWaMIUKTZhChe5CQUGB0BGcirPvDyp0F5z9P9DRnH1/UKEJU+iDFcIUp/7GyqGvzgsdAXOmh+K/z5cLHQOzpowXOgIAwFcshqGhQdAMA6WSDu+jIQdhChWaMIUKTZhChSZMoUITplChCVOo0IQpVGjCFCo0YQoVmjCFCk2YQoUmTKFCE6ZQoQlTqNCEKVRowhQqNGEKFZowhQpNmEKFJkyhQhOmUKEJU6jQhClUaMIUKjRhChWaMIUKTZji1Ne2c6Tau1XYtTnLuqyuq8Vrv38Dbfp/4IMPP8QvNf/AR5ty8PwLYwRMKYx3MzJw9uy3kEqlqKi4JnScTjms0FevXkVeXh44jkNsbCzmz5/vqE3zMnjIcHy8VQkA4MxmZKQsQPi/TsXYsSPx7qo/Y1/OFoETCic5ORlLly5Fenqa0FG65JAhB8dxUCqVyMzMxNatW1FSUoKamhpHbLpHblwrR8CgIZAGDMLo3/0Og4cMFzqSoKKio+Hr5yd0DF4cUmiVSoVBgwYhMDAQIpEI0dHRKCsrc8Sme+Ti9/+DqKnOOVMq6ZxDCq3T6SCR/HZNX4lEAp1O54hNd5vpwQOUl5UgIjpG6CikB5zqRWFBQYF1Do8NGzZgzvRQh2c4ffoUwsPCkPyKAgAg9vbEnOmhyPH1RnToGEyc6PhMAODjNUCQ7T5i8PaGq2s/9OvXD75isaBZOuOQQvv7+0Or1VqXtVot/P39n1pPoVBAoVBYl4W4cv7uz/8LIS9FWLf96Ar+OoMRpeU3ca/R4ZEACH8F/0ajERxnhtlspiv4jxw5ErW1tVCr1TCZTCgtLUV4eLgjNt0tLS33cePqDwiPnGq97dTf/4aMlAVQ3arElvVrsCnrPQETCuOt1D8hbs5sqFQqjBgRjL8eOiR0pA45bNKg8vJy7N+/HxzHQS6XIyEhoct/Q3Os/EboI/Qjzj7HisPG0KGhoQgNFWb8SfoO+uibMIUKTZjS4ZDjgw8+gIuLS5cPkJWV1eU6hDhKh4WeMYM+KSP/fDosdExMjANjENI7eL3LYbFYcO7cOZSUlMBoNGLz5s2orKyEwWBAdHS0vTMSwhuvF4X5+fkoLCyEQqFAfX09gIfnY3z99dd2DUdId/Eq9Pnz57Fq1SpMnjzZ+kIxICAAarXaruEI6S5eheY4Dh4eHja3tbS0PHUbIULjVeiJEyfiwIEDePDgAYCHY+r8/HyEhYXZNRwh3cWr0IsXL4Zer8eSJUvw66+/YvHixdBoNFi0aJG98xHSLbze5fD09MR7772HhoYGaDQaSKVS+Pr62jkaId3H++Sk5uZmVFRUQK/Xw8/PDxMnToSXl5c9sxHSbbwKff36dWzevBlBQUGQSqXQarVQKpVYuXIlJkyYYO+MhPDGq9BKpRKpqak2H6JcuHABSqUS27Zts1c2QrqN14tCvV6PyMhIm9siIiJgMBjskYmQHuNV6GnTpuHMmTM2t3377beYNm2aXUIR0lO8Th/lOA5nz57FyZMn4e/vD51Oh4aGBowaNcphQQnhg/fpo7GxsXYPQ8izotNHCVN4vw9tMBigUqlgNBrx+BfF6YsAxJnwKvTly5exc+dODB48GNXV1Rg2bBiqq6sxZswYKjRxKrwKnZ+fj+XLlyMqKgpvvPEGNm3ahMLCQlRXV9s7HyHdwuttu/r6ekRFRdncNn36dBQVFdklFCE9xavQPj4+1g9RBg4ciNu3b6Ourg4cx9kzGyHdxmvIERsbi5s3byIyMhJz585FVlYWXFxcEB8fb+98hHQLr0I/Pn3E9OnTMW7cOLS0tGDo0KH2ykVIj/To2nZSqbS3cxDSKzos9LJly3g9QE5OTq+FIeRZdXg53crKSl4PIJPJejXQ49Qabdcr2Zmfrxh6g7CXjwWAil8EutL6EyJGBeHyT78ImkHx4ogO7+vwCG3PohJiL3T1UcIUKjRhChWaMKVbheY4Dnq93l5ZCHlmvN6Hbm5uxt69e3Hx4kWIRCIcPHgQV65cgUqlQnJysr0zEsIbryN0bm4uPD09kZ2dDZHo4c/A6NGjUVpaatdwhHQXryP0tWvXsGfPHmuZgYcnLDUIPL0XIU/idYT29PSE0Wi0ua2+vh5+fn52CUVIT/EqdGxsLLZs2YLr16/DYrHg9u3b2L17N2bOnGnvfIR0C68hxyuvvAJ3d3colUqYzWbk5ORAoVAgLi7O3vkI6RZehXZxcUFcXBwVmDg93hdr7Mj48c4xBzUhAM9CP3mKaGNjI0wmEyQSCXbt2mWXYIT0BK9C796922aZ4zgcP34cAwYMsEsoQnqqR+dyuLq6IiEhgaZ1I06nxycnVVRUwNWVzm0izoXXkOPJr2O1tbWhra0NKSkpdglFSE/xKvQ777xjs9y/f38MHjwYnp6edglFSE91WWiO43D06FGsXbsWbm5ujshESI91OQh2dXWFWq1GB9+lJcSp8HpVt2DBAuTm5kKj0YDjOJs/hDgTXmPoPXv2AEC7F2fMz8/v3USEPANehaZPA8k/C15DjgsXLmDgwIFP/bl06ZK98xHSLbwKffz48W7dTohQOh1yPDrLjuO4p864q6uro3M5iNPptNCPzrJra2uzOePOxcUFvr6+ePPNN+2bjpBu6vBijY/btWsX0tPTHZHHhlAXa7x79y7S05ejXqNBv34i/GHRIqSmviVIlkeEulij2WxG2h9egTQgEOt3KhHg1oqkpGQ0Nhgwaux4rPp4C9zc3B2aqbOLNfIaQz9rmbOzs5GSkoKVK1c+0+M4ikjUD1lZf8b3xaUoLi5G3j4lbt26JXQsQXx5OA/DR4y0Ln/4n/+BhNffxP6/FcLLxwdnvjwqYLqnOeR0uZiYGGRmZjpiU70iMHAQQkJeBAB4e3tj1OjRuFdbK3Aqx9PU1eLS94WYk5AEALBYLCg6/x2mKeYAAP5t3msoKTwrZMSnOKTQMpkMXl5ejthUr7tz5w6uX7uG0LAwoaM4XM5f1uFPK1bD1eVhTRoNeojFYvT7/+uzSAMHQauuEzLiU+iE5k40NzUhKSkR69Z9DG9vb6HjONTFonPw9ZNgtGyC0FG6pUdzrNhLQUEBCgoKAAAbNmyAn69YsCwPHjzA64t+j0WLFuH11xcJluORiH9x7G+40wdv4YeS77D05Ri0tLTCaGxEfs5f0NjQgNARARCJRLisq8LzwcMRMSrIodk641SFVigUUCgU1mWhpoKwWCx4Jz0NwSOeR0bGu31ySoq4Py5H3B+XAwD+t+wijh3IRWrmp2h78ABbcpSQz56Hbdl7MC5iqsOnqHjmdzn6msuXLuHYsaMo/v57hIeHYYY8BgUFzvXiRyhZ69bj+EEl/n2eHI0GA2a/mih0JBu83od+Vtu2bUNlZSWMRiPEYjESExN5TXpPkwb9hiYN+k2PJg3qTStWrHDEZgihIQdhCxWaMIUKTZhChSZMoUITplChCVOo0IQpVGjCFCo0YQoVmjCFCk2YQoUmTKFCE6ZQoQlTqNCEKVRowhQqNGEKFZowhQpNmEKFJkyhQhOmUKEJU6jQhClUaMIUKjRhChWaMIUKTZhChSZMoUITpjjkcrqEOAodobuwevVqoSM4FWffH1RowhQqNGEKFboLj09ixFdaWhoqKioAACdOnMDnn3/e27GecuPGDbz99tsd3p+YmIh79+51+ThqtRqJiYkwm83t3t/Z/ujq3zqCU82C5Yx6UujHJSQk8Fpv9+7dkEgkSE5Ofqbt2duz7g97oyN0F4Q82pDu65NH6LS0NCgUChQVFcFgMGDSpElISUmBu7s7bty4gZ07d2L27Nk4deoUQkJCkJaWhpMnT+LcuXNobm7G+PHjkZqaap3uuaioCEeOHEFLSwvi4+NttnX06FHcu3cPGRkZAICbN2/i0KFDqKmpwYABA5CUlASTyYTi4mIAwKlTpzBu3DisXr0aOp0O+/btw48//ggPDw/MnTsXcXFxAIC2tjbk5ubiypUr8PX1hVwu5/38y8vLceTIEdTV1cHT0xNyuRyJibbTsxUWFuLYsWOwWCyIj4/Hyy+/DADgOK7TfSG0PlloACguLsbatWvh4eGBjRs34sSJE9Zf9waDAU1NTcjOzobFYsGZM2dQVlaGjz76CD4+PsjLy8PevXuxYsUK1NTUIDc3F2vWrMGoUaNw+PBhaLXtT0en0WjwySefIDU1FZGRkbh//z60Wi2Cg4Nx69YtmyEHx3HYuHEjJk2ahBUrVkCr1WLdunUICgrCSy+9hGPHjqGurg47d+5ES0sLPv30U97PvX///khPT8fQoUNRXV2N9evXIzg4GBEREdZ1rl+/ju3bt0OtViMrKwvBwcEICQnpdF84gz475Jg1axakUim8vLzw6quvoqSkxHqfi4sLEhMT4ebmBnd3d5w9exbJycmQSCRwc3PDwoULcenSJZjNZly8eBFhYWGQyWRwc3NDUlISXFxc2t1mcXExJkyYgClTpkAkEsHb2xvBwcHtrvvzzz+jsbERCxYsgEgkQmBgIGJjY1FaWgoAuHDhAhISEuDl5QWpVIo5c+bwfu7jxo3D8OHD4erqiueeew6TJ09GZWWlzToLFy6Eh4cHhg8fDrlcbt0/ne0LZ9Bnj9BSqdT694EDB0Kn01mXfXx84O7ubl3WaDTYvHmzTVFdXV3R0NAAnU4HiURivd3Dw6PDie61Wi0CAwN55dNoNNDr9ViyZIn1No7jMHbsWACAXq+32e7jz6crP/30Ew4fPoyqqiqYTCaYTCZERkbarPPkY1dVVVlzdbQvnEGfLXR9fb3N3/39/a3LTx5hJRIJli1bhjFjxjz1OH5+frh79651ubW1FUajsd1tSiQSqFSqdu97cptSqRQBAQHYsWNHu+v7+vpCq9Vi2LBhTz2fruzYsQOzZs3CmjVr4O7uji+++AKNjbYz1Wq1WgwZMsT62H5+ftbn0NG+UKvVvDPYS58dcnzzzTfQarVoamrCiRMnEBUV1eG6M2fOxJEjR6DRaAAAjY2NKCsrAwBERkbihx9+wM2bN2EymZCfn4+OTo+ZOnUqrl27htLSUpjNZhiNRty5cwcAIBaLUVdXZ133hRdewIABA/DVV1+hra0NHMehqqrK+gMRFRWFL7/8Ek1NTdBqtThz5gzv537//n14eXnB3d0dKpXK+oL0ccePH0drayuqq6vx3XffITo6ust94Qz67BF6ypQpWL9+PfR6PcLDw/Haa691uO6jdxYerS8WixEVFYVJkyZh2LBhWLp0KbZv347W1lbEx8fb/Lp+nFQqxZo1a3Dw4EHs2bMHnp6eSEpKQnBwMGbMmIHPPvsMS5YsgUwmw/vvv49Vq1bhwIEDSEtLg8lkQlBQEJKSkgA8HOPm5uYiPT0dfn5+kMvlOH36NK/nnpKSggMHDmDfvn2QyWSIiopCc3OzzToymQwZGRngOA7z5s3Diy++2OW+cAZ98my7tLQ0vPXWWwgJCRE6CullfXbIQdhEhSZM6ZNDDsIuOkITplChCVOo0IQpVGjCFCo0YQoVmjDl/wCeuO9su9rQ7gAAAABJRU5ErkJggg==\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 混淆矩阵：\n",
    "# 加载数据\n",
    "df = pd.read_csv(\"http://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/wdbc.data\",header=None)\n",
    "'''\n",
    "乳腺癌数据集：569个恶性和良性肿瘤细胞的样本，M为恶性，B为良性\n",
    "'''\n",
    "# 做基本的数据预处理\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "X = df.iloc[:,2:].values\n",
    "y = df.iloc[:,1].values\n",
    "le = LabelEncoder()    #将M-B等字符串编码成计算机能识别的0-1\n",
    "y = le.fit_transform(y)\n",
    "le.transform(['M','B'])\n",
    "# 数据切分8：2\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,stratify=y,random_state=1)\n",
    "from sklearn.svm import SVC\n",
    "pipe_svc = make_pipeline(StandardScaler(),SVC(random_state=1))\n",
    "from sklearn.metrics import confusion_matrix\n",
    "\n",
    "pipe_svc.fit(X_train,y_train)\n",
    "y_pred = pipe_svc.predict(X_test)\n",
    "confmat = confusion_matrix(y_true=y_test,y_pred=y_pred)\n",
    "fig,ax = plt.subplots(figsize=(2.5,2.5))\n",
    "ax.matshow(confmat, cmap=plt.cm.Blues,alpha=0.3)\n",
    "for i in range(confmat.shape[0]):\n",
    "    for j in range(confmat.shape[1]):\n",
    "        ax.text(x=j,y=i,s=confmat[i,j],va='center',ha='center')\n",
    "plt.xlabel('predicted label')\n",
    "plt.ylabel('true label')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-14T12:41:31.021663Z",
     "start_time": "2021-03-14T12:41:20.711664Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 0 Axes>"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 504x360 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcQAAAFRCAYAAADq9N3vAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABTMElEQVR4nO3dd0BV9f/H8eeFy5Y9RMSJ4iwXbgUHWmlamqPScn0dobl3mmSuzB2W5UpbkvVVKzMTF25xlTMVRw5EAVGQebmf3x/8vN9IxavCvYz34y/uuYdz3vdzL/fF53M+5xyNUkohhBBCFHMW5i5ACCGEKAgkEIUQQggkEIUQQghAAlEIIYQAJBCFEEIIQAJRCCGEACQQhRm0aNGC//znP+Yuo1AqX74806ZNM3cZXLp0CY1Gw+7du81disns2LEDjUbD1atXn2k7BeU9FA/SmrsAYRq9e/dm1apVAFhYWFCqVClatWrFzJkzKV26tElr+e9//4tWKx+93PznP//h/Pnz7NixI8fyqKgo7O3tzVNUIRMcHIyvry9ffvllnmyvSZMmxMTE4OXlZdT68h4WPtJDLEaaN29OTEwMf//9N99++y1Hjx6la9euJq/Dzc0NJyenfN2HUorMzMx83cfTeNaaPD09cXBwyKNqCqaC+r5ZW1vj7e2NhcWzfW0Wh/ewsJJALEbu/0GXLl2awMBABgwYwL59+7h7965hnS1bttC0aVPs7OwoXbo0ffr0IT4+Psd2wsPDqVevHra2tri7u/PSSy9x+/Ztw/OffPIJVatWxdbWlsqVKzN9+nR0Op3h+X8OmS5duhRnZ2fS0tJy7OOjjz6ibNmy6PV6AM6fP89rr72Gi4sLrq6utG3bluPHjxvW//LLL9FqtWzfvp06depgY2NDRETEQ9shKSmJgQMH4unpiY2NDQEBAfz++++G5+8PB3799de0bt0aOzs7KlasyJo1a3JsJzY2lt69e+Pp6YmjoyNNmzYlMjLS8Pz9IbaNGzfSrFkzbG1tWbZsGbdv36Znz56ULVsWOzs7qlSpwty5c7l/0ajQ0FCWL1/Ozp070Wg0aDQaQy/n38Nt5cuX5/3332fYsGG4ublRsmRJRowYkaO9U1NTGTBgAM7Ozri6uhISEsKECROoVKnSQ9vnvuTkZIYPH06ZMmWwsbGhfPnyzJgxI8c6169f5+WXX8be3p6KFSs+0BtbuHAhtWvXpkSJEnh7e/P6668TExPzzG1036M+i71792br1q2sWrXK0Ib3e2pP+779e8g0MzOTkSNH4uvri42NDaVKleL1119/4vdQp9PxwQcf4Ofnh42NDaVLl+bdd9/N9b0R+USJYqFXr16qdevWhsfXrl1TgYGBytLSUiUnJyullNq6dauys7NTixYtUmfPnlUHDx5ULVq0UIGBgUqv1yullFqxYoXSarVq6tSp6uTJk+qPP/5QCxYsULdu3VJKKTVlyhRVtmxZ9d///ldduHBBbdy4UZUpU0ZNmjTJsO+goCDVr18/pZRSiYmJytbWVq1ZsyZHvdWrV1cTJkxQSil148YNVbJkSTVo0CD1559/qjNnzqghQ4YoNzc3dfPmTaWUUitXrlQajUbVr19fbdu2TUVHRxue+7cuXbqocuXKqd9++02dOnVKDR06VFlZWanTp08rpZS6ePGiAlSpUqXU119/rc6cOaPee+89ZWFhoY4cOaKUUiolJUVVq1ZNde7cWUVFRalz586padOmKWtra3Xq1CmllFLbt29XgKpSpYr66aef1IULF9SVK1dUTEyMmjlzpjp8+LC6cOGC+uqrr5SDg4NasWKFUkqppKQk9eabb6rGjRurmJgYFRMTo1JSUpRSSpUrV059+OGHhtdSrlw55eLiombOnKnOnj2rwsPDlVarVcuWLTOs8+677yovLy+1YcMGdebMGTV+/Hjl5OSk/Pz8Hvl50ev1KigoSFWoUEGtW7dORUdHq507d6ovvvgiRxtVqFBBhYeHq3PnzqkJEyYoS0tL9ddffxm2s2DBArVlyxZ14cIFtXfvXtW4cWMVGBhoeP5p2+hxn8XExETVvHlz1a1bN0MbpqenP9P7dn/5lStXlFJKzZ07V5UuXVpt375dXb58WR08eFDNnz//id/Dt99+W3l6eqrVq1er8+fPq3379ql58+Y98r0R+UcCsZjo1auXsrS0VA4ODsrOzk4BClCjRo0yrBMUFKTGjRuX4/cuX76sAHX06FGllFJlypRRgwcPfug+7t27p+zs7NSmTZtyLF+1apVydnbOsZ/7gaiUUt27d1ft2rUzPI6KilKAOnPmjFIqO2QbNmyYY5t6vV5VrFjR8AW0cuVKBajIyMhc2+HcuXMKUBs3bsyxvE6dOqpPnz5Kqf992f8zxJVSqnHjxqpnz56G/ZUuXVplZmbmWKdly5Zq2LBhSqn/fbGuXr0615qUUmro0KEqODjY8Lhfv34qKCjogfUeFogdOnTIsc6LL76oXn/9daWUUsnJycra2jpHQCqlVMOGDXMNxIiICAWoqKiohz5/v43mzp1rWKbT6VSJEiXUkiVLHrndI0eOKEBdvXpVKfVsbZTbZ1EppVq3bq169eqVY9mzvG//DsShQ4eqli1bGv5Z/Ddj3sP7n8e1a9c+8nUI05GZDcVIw4YNWbVqFWlpaXz//fdERETkGLqJiopi//79hIWFPfC7586dw8fHhytXrtC2bduHbv/kyZOkpqby2muvodFoDMuzsrJIS0vj1q1beHp6PvB7vXr1omPHjty8eRMvLy9Wr15NgwYNqFKliqGuw4cPU6JEiRy/l5qayrlz53Isq1+/fq5tcOrUKQACAwNzLA8MDGTfvn05ljVu3DjH46ZNm7J161ZDTTdu3MDFxSXHOunp6djZ2eVY1qBBgxyP9Xo9s2fPZs2aNVy9epW0tDQyMzMpV65crrU/Su3atXM89vHx4eLFi0D2UHNGRgaNGjV64LX9/PPPj9zm4cOHcXV1JSAgwOh9W1pa4uXlRWxsrGHZjh07mDlzJqdOnSIxMdEwBH758uUck7metI1u3ryZ62fxUZ7lffu3Pn360KZNGypVqkSbNm1o06YNHTp0wNra2uh6jhw5AvDEr0PkDwnEYsTOzs5w3KhmzZpER0fz7rvvsnTpUiD7S2jcuHG89dZbD/yut7c3KSkpuW7//pfd2rVr8ff3f+B5Nze3h/5e27Zt8fDw4Ntvv2Xw4MGsWbOG0NDQHNtt3br1Q4Pa2dnZ8LOlpSW2tra51phX9Ho91apVY926dQ889+8ZhP+eQDF37lxmzpzJ/PnzqVOnDo6OjsyfP5+NGzc+VS3//gLWaDSG9+Kfy/JDbvv++++/adeuHW+99Rbvv/8+Hh4eXL16leDgYDIyMnL8Xn630X3P8r79W+3atbl48SJbtmxh+/btDBs2jMmTJ7N///58nzQm8ocEYjEWGhpKtWrVGDhwIAEBAQQEBHDy5MlHTrYoUaIEvr6+/P7773Ts2PGB52vUqIGtrS0XLlygXbt2RtdhaWlJjx49+Oqrr6hYsSJ37twxTE4ACAgI4Msvv8TX1/eZA69GjRoAREZG5qgxMjKSOnXq5Fh3//79OdbZu3cv1atXN9S0evVqnJycjJ6G/899vfjii/Tt29ew7N89XWtra7Kysp5ouw9TqVIlrK2t2bdvn6F2yH5tualXrx63b9/m0KFDj+0lPkpUVBSpqaksWLDA0Ps6fPiwUb/7uDby8vLK9bMID2/DZ3nfHqZEiRJ06tSJTp06MXHiREqVKsXOnTsNPcXHvYd169YF4Pfff6dLly7PXI94NjLLtBirXLkyHTp04L333gNg6tSpbNiwgZEjR3Ls2DGio6P57bff6NevH6mpqQBMmTKFzz//nA8//JDTp09z8uRJwsLCiIuLo0SJEkycOJGJEyeyePFi/vrrL06ePMmaNWsYN25crrW8/fbbHDlyhClTpvDyyy/n6E0OGTKErKwsXnnlFXbt2sWlS5fYvXs37733Hnv37n2i1+zn50fXrl0JCQlh8+bNnDlzhmHDhnHixAnGjBmTY93ly5fz7bffcvbsWd5//3327dvHyJEjAejRowcVKlSgffv2/P7771y6dIkDBw4wc+ZM1q9fn2sNVapUYceOHWzfvp2zZ88yadIkDhw4kGOdChUqcObMGU6ePElcXBzp6elP9Drvc3BwYODAgUyaNIlffvmFs2fP8t5773H69Olce42tWrWiefPmdO/enQ0bNnDx4kX27NnDsmXLjN535cqV0Wg0zJ07l4sXL7J+/XqmTp1q1O8a00a5fRYhuw0PHz5MdHQ0cXFxZGZmPtP79m8ff/wx33zzDSdPnuTixYusWLECS0tLw+iIMe9hpUqV6NGjByEhIXz99ddER0cTFRXFwoULn6gWkUfMfRBTmMa/Z5net2fPHgWo7du3K6WUioyMVK1bt1YlSpRQ9vb2qmrVqmrYsGE5JiF8/fXX6vnnn1fW1tbKzc1NtWvXTt2+fdvw/NKlS1WtWrWUjY2NcnFxUQ0aNFCffvqp4fl/T6q5r3bt2gpQ69evf+C5S5cuqTfffFN5eHgoa2trVbZsWdWjRw914cIFpVT2ZAlLS0uj2uLOnTtqwIABhm3Vq1dPbd682fD8/Qkjq1evVkFBQcrGxkaVL19effPNNzm2ExcXpwYNGqR8fHyUlZWV8vHxUa+++qphJuq/J2Hcl5iYqLp27aocHR2Vm5ubCgkJUZMmTVLlypUzrBMfH69eeukl5eTkpAC1cuVKpdTDJ9X887FSD07mSElJUf3791eOjo7K2dlZvfPOO2rYsGGqZs2aubbT3bt31ZAhQ5S3t7eysrJS5cuXVzNnzszRRrt27crxO35+fmrKlCmGx2FhYcrX11fZ2tqqpk2bqk2bNuX4vD1LGymV+2cxOjpaNW/eXDk4OOTY59O+b/9evmTJElW3bl3l6OioHBwcVEBAQI7PrrHvYUZGhuG1WVlZqdKlSxsm+AjT0ij1rxN7hCjmLl26RIUKFdi1axfNmjUzdzn5olWrVri6uvLjjz+auxQhCgw5hihEEXf8+HGOHDlC48aNycjI4KuvvmL79u1s2rTJ3KUJUaBIIApRxGk0Gj777DOGDh2KXq+natWqrFu3jhdffNHcpQlRoMiQqRBCCIHMMhVCCCEACUQhhBACkEAUQgghgCIwqeb69et5sh0PDw/DCb3i0aSdjCPtZDxpK+NIOxnPx8fnqX5PeohCCCEEEohCCCEEIIEohBBCABKIQgghBCCBKIQQQgASiEIIIQQggSiEEEIAJjoP8dNPP+XIkSM4Ozszd+7cB55XSrFy5UqOHj2KjY0NISEhVKxY0RSlCSGEEICJeogtWrRg4sSJj3z+6NGj3Lhxg0WLFjFgwIAnuiu3EEIIkRdM0kOsXr06N2/efOTzhw4dIjAwEI1Gg7+/P/fu3eP27du4urqaojyTcdv5FrbXt5m7jGf2dNeAKH6knYwnbWUcaafcXU5wRq80MOP2U/1+gbh0W0JCAh4eHobH7u7uJCQkPDQQIyIiiIiIAGDWrFk5fu9ZaLXaPNvWo1gXgTAUQoiCSCnotOp17K0y2T3j6bZRIALxSQQHBxMcHGx4nFfX9jPFdQLv/3d3/Y1r+bqf/CTXUzSOtJPxpK2MI+30eJP9Yli+/ORT/36BCEQ3N7ccb3R8fDxubm5mrEgIIURBptcrvvrqNJcu3WXKlEYANGxYioYNSz31NgvEaRcBAQFERkailOLs2bPY29sXueOHQggh8sbly3fp1m0jEyfu4YsvjnPyZHyebNckPcQFCxZw6tQpkpKSGDRoEN26dUOn0wHQtm1b6tSpw5EjRxg6dCjW1taEhISYoiwhhBCFiF6vWLXqFNOnHyQ1VYe7uy0zZjSlRg33PNm+SQJx+PDhuT6v0Wj4z3/+Y4pSjFZUZoQKIURRcOnSXUaPjmTfvhgAXnnFjw8/bIy7u12e7aNAHEMsiPIrDNN8WuXLdoUQoigLCzvGvn0xeHjYMXNmU9q1q5Dn+5BAfIzCPCNUCCEKM71eYWGhAeC99xpgZWXBmDEBuLnZ5sv+CsSkGiGEEOI+vV6xdOlxOnTYQHp6FgCurrbMnNks38IQpIcohBCiAImOTmTUqEiiomIB2Lz5Eh07+plk3xKIQgghzC4rS8+yZSeYPfsQaWlZeHnZ8dFHzWnbtpzJapBAJHtGqfX1bXKdQCGEMIPz57N7hYcOZfcKu3SpTGhoI1xd82949GEkEHn0jFKZESqEEPnvyJGbHDoUS8mS9nz0UTPatDFdr/CfJBD/QWaUCiGEaSQnZ1CihDUAXbtWJjExnW7d/HFxsTFbTTLLVAghhMlkZelZsuRPGjZcw/nziUD2xVkGDHjOrGEI0kMUQghhIufO3WbkyEiOHMm+P+6WLZepVMnFvEX9gwSiEEKIfKXT6fnii+PMmXOY9PQsvL0d+Pjj5rRqVcbcpeUggSiEECLfREcnMmzYDo4evQXAG29U4f33G+HkZG3myh4kgSiEECLfKAWnTydQqpQDc+Y0p0WLgtUr/CcJRCGEEHnq8uW7lC3riEajoVIlF1aubEvt2l4Fslf4TzLLVAghRJ7IzNSzcOFRgoLW8v335wzLAwN9C3wYgvQQhRBC5IHTpxMYMWInx4/HAXD27G0zV/TkJBCFEEI8tcxMPWFhx1i48CiZmXpKly7BnDnNCQz0NXdpT0wCUQghxFO5fj2ZPn1+58SJeADeeqsakyY1MFyBprCRQBRCCPFU3N3tyMzU4+tbgjlzAmnevLS5S3omEohCCCGMdvJkPD4+Dri62mJjY8ny5W3w9LQrtL3Cf5JZpkIIIR4rIyOLuXMP067dOqZM2WdYXqGCc5EIQ5AeohBCiMc4cSKOESN2cupUAgBOTtZkZemxtCxafSoJRCGEEA+VkZHFwoVHCQs7hk6nKFfOkblzg2jcuJS5S8sXEohCCCEekJqqo0OHDZw+nd0r7NevBuPH18fe3srMleUfCUQhhBAPsLPTUq+eF6mpOubNC6Rhw6LZK/wnCUQhhBAA/PHHLbKyFHXregHw/vuNsLDQYGdXPKKieLxKIYQQj5SensW8eUf47LM/KFPGkS1bOmNvb4WDQ9EdHn0YCUQhhCjGjh69yciROzl7NhGNBtq2LYdGozF3WWYhgSiEEMVQWpru/3uFf6LXKypWdGbevCDq1y9p7tLMRgJRCCGKGaUUPXv+xr59MWg0MGjQ84weXa/YHCt8lOL96oUQohjSaDT06lWdmzdTmDcviICA4tsr/CcJRCGEKAYOHYrlzJkEevasBkCHDhV54YVyWFtbmrmygkMCUQghirDUVB0ff3yIL744jlZrQUBASapWdQOQMPwXCUQhhCiioqJiGTlyJxcu3MHCQsPAgc9RvryTucsqsCQQhRCiiElN1TF79iGWLj2OUuDv78K8eUHUqeNl7tIKNAlEIYQoYiZP3st33/2FpaWGkJBajBhRFxsbGR59HAlEIYQoYoYNq8Nff91m2rQm1Krlae5yCo2idTMrIYQohg4ciGHkyJ3o9QqAMmUc+emnjhKGT0h6iEIIUUilpGQya1YUK1acRClo0sSHLl0qAxTby689CwlEIYQohPbvj2HUqEguXbqLVqthyJDadOxY0dxlFWomC8Rjx46xcuVK9Ho9rVu35tVXX83xfFxcHIsXL+bevXvo9XrefPNN6tata6ryhBCiULh3L5OZMw+ycuUpAKpVc2PBgiBq1vQwc2WFn0kCUa/Xs3z5ciZNmoS7uzsTJkwgICAAX19fwzo//vgjjRs3pm3btly9epWZM2dKIAohxL+sWfMXK1eeQqvVMHRoHd59t7acYJ9HTBKI58+fx9vbm5Ils6+X16RJE6KionIEokajISUlBYCUlBRcXV1NUZoQQhR4SinDz716Vef48Tj+85/nqFnT3YxVFT0mmWWakJCAu/v/3jh3d3cSEhJyrNO1a1d27drFoEGDmDlzJn379jVFaUIIUaDt3n2N9u3XExt7DwCt1oIFC1pIGOaDAjOpZs+ePbRo0YIOHTpw9uxZPvnkE+bOnYuFRc7MjoiIICIiAoBZs2bh4ZF34+Z5ua2iSqvVSjsZQdrJeNJWD5eUlM6ECdtZuvQoAAsXRjFjRgvzFlXEmSQQ3dzciI+PNzyOj4/Hzc0txzrbtm1j4sSJAPj7+5OZmUlSUhLOzs451gsODiY4ONjwOC4u7pnr88nDbRV1Hh4e0k5GkHYynrTVgyIjrzFmTCRXryZjZWXBsGF1CA1tLu1kJB8fn8ev9BAmGTL18/MjJiaGmzdvotPp2Lt3LwEBATnW8fDw4MSJEwBcvXqVzMxMnJzkIrRCiOIjKSmDsWN38cYbv3L1ajLPPefBr7++yogRdbGykokz+c0kPURLS0v69u3L9OnT0ev1tGzZkjJlyhAeHo6fnx8BAQG8/fbbfP7552zcuBGAkJAQObFUCFGs/PXXbb799gxWVhaMGFGXkJBaWFnJBcVMRaP+OX2pELp+/fozb8Pnu9LZ23rj2jNvq6iT4S3jSDsZr7i3VXp6Vo4Lb69ceZLGjUsZ7ll4X3FvpydRoIdMhRBCPGjHjis0axbOzp1XDcv69KnxQBgK05BAFEIIE7t7N4PRoyPp0eM3rl+/x9dfnzZ3SYICdNqFEEIUB9u2XWHs2F3ExNzDxsaS0aPrMWDAc+YuSyCBKIQQJnH3bgahofsIDz8LQJ06XsyfH0jlynJVroJCAlEIIUxAr1fs3HkVGxtLxo4NoH//mlhaylGrgkQCUQgh8kliYjq2tpbY2mpxcbFh8eJWeHjYUamSi7lLEw8h/54IIUQ+2LLlMq1a/cC8eUcMyxo1KiVhWIBJD1EIIfLQ7dtpTJmyjx9/PA/AoUOx6HR6tFrpfxR0EohCCJFHfv/9MuPG7eLmzVRsbS0ZN64+/frVkGOFhYQEohBCPKO0NB1jxuziv//N7hXWr1+SefOCqFjR+TG/KQoSCUQhhHhGNjaW3L2bga2tJRMmNKBPn+rSKyyEJBCFEOIpJCSkkZycQdmyTmg0GmbPbs69e5nSKyzE5F8YIYR4Qr/+epGWLX8gJGQ7WVl6AEqWtJcwLOQkEIUQwkgJCWm8885W+vePIC4ue+LMnTsZ5i5L5BEZMhVCCCNs3HiRCRN2Ex+fhr29lvfea8Dbb1fHwkLu21pUSCAKIcRjjBixk++/z74GaePGpZg7N5By5ZzMXJXIaxKIQgjxGNWru2Fvr2XSpIa89VY16RUWUUYH4p9//smePXu4c+cO48ePJzo6mtTUVGrWrJmf9QkhhMnFxaVy+nQCzZuXBqBv3xq0a1eB0qVLmLkykZ+MmlSzadMmli5dSqlSpTh9OvtGltbW1qxZsyZfixNCCFNSSrFhQzQtW/5A//5buHYtGQBLSwsJw2LAqB7ir7/+yuTJk/Hy8mLDhg0AlC5dmuvXr+drcUIIYSq3bqUwceIefv31EoChdyiKD6MCMTU1FQ8PjxzLdDodWq0cghRCFG73e4WTJu3l9u10SpSw4v33G/Hmm1XQaORYYXFi1JBptWrVWL9+fY5lmzZtokaNGvlRkxBCmMyMGQcZPHg7t2+nExRUmm3butCjR1UJw2LIqEDs27cvBw8eZPDgwaSlpTFs2DD27dtHr1698rs+IYTIVx07+uHiYsOcOc355puX5FhhMWbUmKerqyszZ84kOjqaW7du4e7uTqVKlbCwkAvdCCEKl9jYFH76KZr+/Z8D4LnnPIiKegN7eyszVybMzahEmz17NhqNhkqVKtG4cWP8/f2xsLBgzpw5+V2fEELkCaUUP/xwjpYt1xIaup9Nmy4anpMwFGBkD/HkyZNPtFwIIQqSGzfuMW7cbiIi/gagZUtfatXyNHNVoqDJNRDDw8OB7Bml93++LzY2Fk9P+UAJIQoupRRr154jNHQfd+5k4ORkTWhoI7p185dJM+IBuQZifHw8AHq93vDzfR4eHnTr1i3/KhNCiGe0evVpJk7cA0CrVmX46KNm+PjIpBnxcLkGYkhICAD+/v4EBwebpCAhhMgrXbpU5ttvz9CvX026dq0svUKRK6OOId4Pw9TUVJKSklBKGZ4rWbJk/lQmhBBP6Pr1ZObPP0JoaGMcHKxwcLBi06ZOcjFuYRSjAvHq1assWrSIy5cvP/Dcv48tCiGEqSmlCA8/S2joPpKSMnFysmHy5IYAEobCaEaddrFs2TJq1KjBihUrsLe3Z+XKlbRp04bBgwfnd31CCJGra9eS6dnzN0aNiiQpKZO2bcsxYMBz5i5LFEJGBeLly5fp0aMHDg4OKKWwt7enZ8+e0jsUQpiNUopvvz1Dq1Y/sGPHVVxcbAgLa8mKFW0oWdLe3OWJQsioIVMrKyuysrLQarU4OjoSFxeHg4MDycnJ+V2fEEI81P79NxgzZhcAL75Yjpkzm+HlJUEonp5RgVi1alX27dtHixYtaNSoETNmzMDKykou7i2EMJvGjUvx9tvVaNSoFB07VpQZpOKZGRWII0eONPz8xhtvUKZMGdLS0ggKCsq3woQQ4p+uXk1iwoQ9jBsXQM2a2bejmzmzmZmrEkXJE9/Q0MLCgsDAQHQ6HREREbz44ov5UZcQQgDZxwq/+uo006Yd5N69TDIysggPb2/uskQR9NhAPH78OJcuXcLb25v69euTlZXF5s2b2bBhAyVKlJBAFELkm7//vsvo0bvYs+c6AO3aVWDGjCZmrkoUVbkG4vr16/nxxx8pU6YMV65c4YUXXuDkyZNYWVkxcOBA6tata6o6hRDFiF6vWL36NNOnHyAlRYebmy0zZjSlQ4eK5i5NFGG5BmJERAQffPABFStW5OzZs0yePJm3336b9u1luEIIkX9u3Upl5syDpKTo6NChItOnN8Hd3c7cZYkiLtdATEpKomLF7P/I/P39sbKyol27dk+1o2PHjrFy5Ur0ej2tW7fm1VdffWCdvXv3snbtWjQaDeXKlWPYsGFPtS8hROGj12dfEtLCQkPJkvbMnNkMa2sLXn5ZeoXCNB57DFEpZbh2qZVV9k009Xq94XkLi8ef26/X61m+fDmTJk3C3d2dCRMmEBAQgK+vr2GdmJgY1q9fz4cffkiJEiW4c+fOE78YIUThdOnSXUaN2kn79hXo27cmAJ07VzJzVaK4yTUQ09LSeP3113Ms+/djY65Wc/78eby9vQ0XAm/SpAlRUVE5AnHr1q288MILlCiRfWsWZ2dn416BEKLQ0usVYWFRTJq0g9RUHdev3+Ott6pjZWXURbSEyFO5BmJYWFie7CQhIQF3d3fDY3d3d86dO5djnevXs2eRTZ48Gb1eT9euXaldu3ae7F8IUfBcvHiHUaMiOXDgBgCdOvkxdWoTCUNhNrkGoqenp6nqQK/XExMTw5QpU0hISGDKlCnMmTMHBweHHOtFREQQEREBwKxZs/Dw8MizGvJyW0WVVquVdjKCtNOj6fWKxYsPMXlydq+wZEkHPvnkRV55xd/cpRVo8pnKf098Yv7TcHNzIz4+3vA4Pj4eNze3B9apXLkyWq0WLy8vSpUqRUxMDJUq5TyOEBwcnONmxXFxcc9cn08ebquo8/DwkHYygrTTo+l0er7++g9SU3V07lyJxYtfRq+/J+31GPKZMp6Pj8/jV3oIk4xN+Pn5ERMTw82bN9HpdOzdu5eAgIAc6zRo0ICTJ08CcPfuXWJiYuTmw0IUEVlZepKSMgDQai2YPz+IlSvb8sknLXFzk9MpRMFgkh6ipaUlffv2Zfr06ej1elq2bEmZMmUIDw/Hz8+PgIAAatWqxR9//MGIESOwsLCgZ8+eODo6mqI8IUQ+On8+kVGjInFzs2XFijZoNBr8/V3x93c1d2lC5KBR98+pMEJcXBwJCQn4+xecsf77k3Gehc93pbO39ca1Z95WUSfDNsaRdsruFS5deoKPPz5EWloW3t72bNz4Kt7eOecFSFsZR9rJeE87ZGpUDzEuLo6FCxdy6dIlAL766iv279/PsWPHGDRo0FPtWAhRdJ0/n8iIETs5cuQmAN26+TNlSiNcXGzMXJkQj2bUMcQvvviCOnXqsGrVKrTa7Ax9/vnn+fPPP/O1OCFE4fP553/Stu1/OXLkJt7eDqxe/QLz5wdJGIoCz6hAPH/+PK+++mqOq9LY29uTkpKSb4UJIQqnW7dSSU/P4vXX/dm27TVaty5r7pKEMIpRQ6bOzs7cuHEjx7js1atX5ZwYIQQ6nZ4rV5KoUCH76lKjRtUjKMiX5s1Lm7kyIZ6MUT3EDh068NFHH7F9+3b0ej27d+9m/vz5vPLKK/ldnxCiAPvrrwReeeUnunTZyJ076QDY2WklDEWhZFQPsVWrVjg6OhIREYG7uzuRkZF0796dBg0a5Hd9QogCSKfT8+mnfzB//hEyMvSUKuXAlSvJODvLcUJReBkViHq9nvr161O/fv38rkcIUcCdOZPAiBE7+fPP7FMAevSoyqRJDXFysjZzZUI8G6MCsX///jRu3JhmzZpRtWrV/K5JCFFArVp1iilT9pGZqcfHx4E5cwIJCvJ9/C8KUQgYFYiTJk1iz549LFy4EAsLC5o2bUqzZs0oW1ZmjwlRnJQp40hmpp4ePaoyeXJDHB2lVyiKjie6Ug3AqVOn2L17NwcOHMDV1ZU5c+bkV21GkSvVmJZcLcM4RaWdMjP17NsXQ2Dg/ybJnD17O08vu1ZU2iq/STsZz2QX9/bx8cHX1xcPDw9u3br1VDsVQhR8p07F8/LL6+nRYxOHDsUalss1SEVRZdSQ6b179zhw4AC7d+/m3LlzPP/887zyyisP3LFCCFH4ZWbqCQs7xoIFR9DpFGXKlOAJB5KEKJSMCsSBAwdSpUoVmjVrxqhRox64aa8Qomg4cSKekSN3cvJk9v1Le/euzsSJDXBwsDJzZULkP6MC8ZNPPsHVVYZJhCjKfvopmnff3Y5Opyhb1pG5cwNp0uTpjsUIURg9MhBPnTpF9erVAbh27RrXrj18wknNmjXzpzIhhEk1alQKR0drOnWqxIQJ9bG3l16hKF4eGYjLly9n7ty5AHz22WcPXUej0RAWFpY/lQkh8lVGRhbffnuGnj2rodVa4OVlz65d3XB1tTV3aUKYxSMD8X4YAixevNgkxQghTOPPP28xcmQkp08nkJycyZAhtQEkDEWxZtRpF7Nnz37ocnOfgyiEeDLp6Vl89FEUL7+8gdOnEyhf3omAgJLmLkuIAsGoSTUnT558ouVCiILnjz9uMWLETv766zYaDfznPzUZP74+dnZGfQ0IUeTl+pcQHh4OgE6nM/x8X2xsLJ6envlXmRAizxw+HEunTj+TlaWoUMGJefOCaNDA29xlCVGg5BqI8fHZ5yLp9XrDz/d5eHjQrVu3/KtMCJFn6tTxomFDb2rW9GDs2ADpFQrxELn+VYSEhADg7+9PcHCwSQoSQjy7tDQdCxcepUePqvj6OmJhoeG779qh1T7x1RqFKDYeGYg3b97Ey8sLgOeee47Y2NiHrleypByQF6IgOXLkJiNG7OT8+USOH4/j669fApAwFOIxHhmIo0ePZvXq1QAMHTr0kRv497FFIYR5pKbqmDv3MJ9/fhy9XlGpkgsjRtQ1d1lCFBqPDMT7YQgSekIUdIcOxTJy5E6io+9gYaEhJOR5Ro2qh62tHCsUwlhP9dcSGxuLRqMxDKkKIczn1q0UunXbSHp6FpUruzBvXhB168rfphBPyqhAXLBgAS+99BJVqlRh+/btLFu2DAsLC/r06UOrVq3yu0YhRC48Pe15993apKXpGDGirvQKhXhKRv3lnDhxgiFDhgDwyy+/MHnyZBwcHPj4448lEIUwsdRUHbNnHyIgoCTt21cAkGOFQuQBowJRp9Oh1WpJSEggOTmZqlWrAnDnzp18LU4IkdPBgzcYOXInFy/eZcOGaIKDy2JjY2nusoQoEowKxPLly7Nu3Tpu3bpF3brZ/4kmJCRgZ2eXr8UJIbKlpuqYNSuK5ctPoBRUrerKvHlBEoZC5CGjTkwaNGgQf//9NxkZGXTv3h2As2fP0qxZs3wtTggBBw7EEBz8I8uWncDCQsOwYXX49ddO1Koll04UIi8Z1UP09vZm2LBhOZY1atSIRo0a5UtRQohsOp2ekSMjuXTpLtWquTFvXiDPPy9BKER+MHo62vbt24mMjCQhIQE3NzcCAwNp2bJlftYmRLGllEKj0aDVWjBnTiC7d19j2LA6WFvLEKkQ+cWoQPzvf//Lzp076dChAx4eHsTFxfHTTz9x+/ZtOnfunN81ClFs3LuXyYwZBwGYPr0pAI0bl6Jx41LmLEuIYsGoQNy6dSuhoaE5bvdUq1YtpkyZIoEoRB7Zs+c6o0bt5MqVZKysLAgJqUXp0iXMXZYQxYZRgZieno6Tk1OOZY6OjmRkZORLUUIUJ/fuZTJt2gFWrz4NQPXqbsyf30LCUAgTM2qWae3atVm0aBHXr18nIyODa9euERYWRq1atfK7PiGKtF27rtG69Q+sXn0arVbD6NH1+PXXTtSs6W7u0oQodozqIfbt25cVK1YwevRosrKy0Gq1NG7cmD59+uR3fUIUaeHhf3HlSjI1a7ozb14QNWpIEAphLo8NxJSUFG7cuEG/fv0ICQkhKSkJR0dHLCzk3mpCPI3UVJ3hjvVTpzahenV3+vd/Disr+ZsSwpxyDcQjR44wf/58MjIysLW1ZcyYMdSsWdNUtQlRpCQlZTBt2gEOH77Jr7++irW1JW5utoSEyKEHIQqCXP8lDQ8Pp0ePHqxevZru3buzZs0aU9UlRJESGXmV1q1/5OuvzxAdncjhwzfNXZIQ4l9yDcTY2FhefPFFbGxseOGFF7hx48ZT7+jYsWMMGzaMd999l/Xr1z9yvf3799OtWzeio6Ofel9CFBR372Ywduwu3nhjE9euJVOrlgebNnWS8wqFKIByHTJVShl+trS0JCsr66l2otfrWb58OZMmTcLd3Z0JEyYQEBCAr69vjvVSU1PZtGkTlStXfqr9CFGQREZeZeTISGJi7mFtbcGoUfUYNOh5tFo5VihEQZRrIKanpzNlyhTD47S0tByPAT744IPH7uT8+fN4e3tTsmRJAJo0aUJUVNQDgRgeHs4rr7zCTz/9ZPQLEKKgun79HjEx96hTx5N584Lw93c1d0lCiFzkGoiDBg3K8fhpr12akJCAu/v/ppO7u7tz7ty5HOtcuHCBuLg46tatK4EoCq1r15INJ9R37+6PnZ2W9u0rSK9QiEIg10Bs0aKFSYrQ6/WsXr2akJCQx64bERFBREQEALNmzcLDwyPP6sjLbRVVWq1W2ukhEhPTGDt2K99/f4qoqH54e2vx9PSkXz+5M8XjyGfKONJO+c/ou108Czc3N+Lj4w2P4+PjcXNzMzxOS0vjypUrhuHXxMREZs+ezdixY/Hz88uxreDgYIKDgw2P4+Linrk+nzzcVlF3/+Lu4n8iIv5m3Lhd3LiRgo2NJZGR56hc2U3ayUjymTKOtJPxfHx8Hr/SQ5gkEP38/IiJieHmzZu4ubmxd+9ehg4danje3t6e5cuXGx6Hhoby1ltvPRCGQhQkiYnphIbuY+3a7OH/unW9mD8/iEqVXMxbmBDiqZgkEC0tLenbty/Tp09Hr9fTsmVLypQpQ3h4OH5+fgQEBJiiDCHyzP79MYSEbCM2NgVbW0vGjAmgf/+aWFrKsUIhCiuTBCJA3bp1qVu3bo5l3bt3f+i6oaGhJqhIiKfn4WFHYmI6AQElmTs3UHqFQhQBRgViZmYmP/zwA3v27CEpKYlVq1bxxx9/EBMTw4svvpjfNQpRIERFxRIQ4IVGo6FSJRfWretAzZru0isUoogw6i951apVXLlyhaFDh6LRaAAoU6YMv//+e74WJ0RBkJCQxrvvbufVV3/ixx/PG5bXquUpYShEEWJUD/HgwYMsWrQIW1tbQyC6ubmRkJCQr8UJYW6//XaJ8eN3c+tWKra2lqSnP93VmoQQBZ9RgajVatHr9TmW3b17F0dHx3wpSghzS0hIY/Lkvaxfn31N3UaNvJkzJ5AKFZzNXJkQIr8YNd7TqFEjwsLCuHkz+wr9t2/fZvny5TRp0iRfixPCHE6fTqBlyx9Yvz4aOzst06Y1Ye3alyUMhSjijArEN998Ey8vL0aNGkVKSgpDhw7F1dWVrl275nd9QphchQpOuLra0LhxKbZufY0+fWpgYaExd1lCiHxm9JBp79696d27t2Go9P6xRCGKgk2bLtK4sQ8uLjbY2moJD2+Pp6edBKEQxYhRgRgbG5vjcWpqquHn+3ewEKIwio9PZeLEPfzyy0W6dq3MggUtAChZ0t68hQkhTM6oQPznZdb+LTw8PM+KEcKUfv75AhMn7iEhIQ17ey116nihlJLRDyGKKaMC8d+hl5iYyNq1a6lWrVq+FCVEfoqLy+4Vbtx4EYCmTX2YOzeQMmVk1rQQxdlTnVXs4uJC7969+fbbb/O6HiHyVUJCGi1b/sDGjRdxcLBi1qxmhIe3kzAUQjz9tUyvX79Oenp6XtYiRL5zc7PlxRfL8fffScyZI71CIcT/GBWI77//fo7jKunp6Vy5coUuXbrkW2FC5AWlFOvXR+Pr60j9+tkTwD78sAk2NpZyrFAIkYNRgdiqVascj21tbSlXrhylSpXKl6KEyAuxsSlMmLCbzZsvU6GCE1u2vIadnRZbW5Pd5EUIUYg89ptBr9dz4sQJBg4ciJWVlSlqEuKZKKX473/P8/77+0hMTKdECStCQmpha2tp7tKEEAXYYwPRwsKCP//8U4aXRKFw48Y9xo/fzZYtfwPQooUvs2c3p3TpEmauTAhR0Bk1y7R9+/Z8//336HS6/K5HiKeWlaWnS5df2LLlbxwdrZg7N5Cvv35RwlAIYZRce4i7d++mWbNm/PbbbyQmJrJx40acnJxyrPPZZ5/la4FCGMvS0oIxYwL44YdzfPRRM3x8JAiFEMbLNRCXLl1Ks2bNePfdd01VjxBGU0qxdu057txJp3//5wDo2LEiHTtWlCF+IcQTyzUQlVIAVK9e3STFCGGsmJh7jB27i23brmBlZUHbtuUoV85JglAI8dRyDcT7M0xzU7NmzTwtSIjcKKX4/vuzhIbu5+7dDJydrZk6tQlly8oJ9kKIZ5NrIGZmZrJkyRJDT/HfNBoNYWFh+VKYEP92/XoyY8fuYvv2qwC0aVOWWbOa4e3tYObKhBBFQa6BaGtrK4EnCoxJk/ayfftVXFxs+PDDJnTq5CdDpEKIPCOX7BCFRmhoI2xttUyZ0kjuVyiEyHO5nof4qKFSIfKbUopvvz1Dv36/o9dnfw7LlnXi009bSRgKIfJFrj3E1atXm6oOIQyuXUtm9OhIIiOvAbB9+xVaty5r5qqEEEWdDJmKAkMpxTffnOHDDw+QnJyJi4sN06c3oVWrMuYuTQhRDEggigLhypUkxozZxa5d2b3Cdu3KM2NGUzw9ZXhUCGEaEoiiQFi37jy7dl3D1dWG6dObytVmhBAmJ4EozCYzU4+VVfa8rnfeqUVSUgYDBz6Ph4edmSsTQhRHRt3tQoi8pNcrvvzyFIGB3xMfnwqAlZUF773XUMJQCGE2EojCpC5fvku3bht57709/P13EuvXR5u7JCGEAGTIVJiIXq9YteoU06cfJDVVh4eHHTNmNKV9+wrmLk0IIQAJRGECly7dZdSonezffwOAV1/148MPm+DmZmvmyoQQ4n8kEEW+i429x4EDN/D0tGPmzKa89JL0CoUQBY8EosgXCQlphh5gw4alWLSoJS1a+EqvUAhRYMmkGpGn9HrF0qXHadDgO8Ol1wA6d64kYSiEKNCkhyjyTHR0IqNGRRIVFQvArl1XCQwsbeaqhBDCOBKI4pllZelZtuwEs2cfIi0tCy8vOz76qDlt25Yzd2lCCGE0CUTxTK5dSyYkZBuHDmX3Crt0qUxoaCNcXWV4VAhRuJgsEI8dO8bKlSvR6/W0bt2aV199Ncfzv/zyC1u3bsXS0hInJyfeeecdPD09TVWeeEr29lr+/vsuJUva89FHzWjTRnqFQojCySSBqNfrWb58OZMmTcLd3Z0JEyYQEBCAr6+vYZ3y5csza9YsbGxs+P333/n6668ZMWKEKcoTT+j8+UTKlHHExsYSV1dbvvzyBcqVc8LFxcbcpQkhxFMzySzT8+fP4+3tTcmSJdFqtTRp0oSoqKgc69SsWRMbm+wv1MqVK5OQkGCK0sQT0On0zJmzn7Zt/8v8+UcMy2vV8pQwFEIUeibpISYkJODu7m547O7uzrlz5x65/rZt26hdu7YJKhPGOnv2NiNH7uTo0VtA9nmGSim5RZMQosgocJNqIiMjuXDhAqGhoQ99PiIigoiICABmzZqFh4dHnu07L7dVVOh0eubPP8DUqbvIyMjC19eJTz99kRde8DN3aQWaVquVz5ORpK2MI+2U/0wSiG5ubsTHxxsex8fH4+bm9sB6f/75J+vWrSM0NBQrK6uHbis4OJjg4GDD47i4uGeuzycPt1WUJCam06PHJo4dy+4VvvlmFRYsaEdmZrK01WN4eHhIGxlJ2so40k7G8/HxefxKD2GSY4h+fn7ExMRw8+ZNdDode/fuJSAgIMc6Fy9eZOnSpYwdOxZnZ2dTlCUew9nZGjc3W3x8HPjmmxf5+ONAnJ3ldAohRNFkkh6ipaUlffv2Zfr06ej1elq2bEmZMmUIDw/Hz8+PgIAAvv76a9LS0pg3bx6Q/d/QuHHjTFGe+IczZxKwtrakYkVnNBoN8+cHYW1tiZOTtblLE0KIfKVRSilzF/Esrl+//szb8Pku+/Ji19+49pg1i67MTD2ffvoH8+cf4fnnPVi3rgOWlg8OIMiwjXGknYwnbWUcaSfjPe2QaYGbVCNM79SpeEaOjOT48ew/turV3cnI0GNnJ9d+F0IUHxKIxVhmpp6wsGMsXHiUzEw9vr4lmDMnkObN5YLcQojiRwKxmNLrFV26/GK4BmmvXtWZOLE+JUrIsUIhRPEkgVhMWVhoaNeuPDdvpjBnTiBNmz7dmLsQQhQVEojFyIkT8Vy/nmy4LdN//lOTnj2r4eDw8HM+hRCiOJFALAYyMrJYtOgYn3xyFHt7K7Zt60KpUg5YWlrg4CATZ4QQAiQQi7wTJ+IYPnwnp09nXyz9tdcqyTmFQgjxEBKIRVR6ehYLFx4lLOwYWVmKcuUcmTs3iMaNS5m7NCGEKJAkEIuo4cN38NNPFwDo168G48fXx95ejhUKIcSjSCAWUYMGPc+pUwnMnt2Mhg2lVyiEEI8jMyqKiD/+uMWcOYcNj2vV8mTbttckDIUQwkjSQyzk0tOzmDfvCJ999gdZWYp69bxo2bIMwEOvRSqEEOLhJBALsaNHbzJy5E7Onk1Eo4GBA5+jUSPpEQohxNOQQCyE0tJ0/98r/BO9XuHn58zcuUHUr1/S3KUJIUShJYFYCIWF/cHixX9gYaHhnXeeZ9SoetjZyVsphBDPQr5FC6GBA5/7/+HSutSrJ71CIYTICzLrohA4dCiWt976jZSUTAAcHa355puXJAyFECIPSSAWYKmpOqZO3c+rr/7Etm1XWLr0hLlLEkKIIkuGTAuoqKgbjBwZyYULd7Cw0DB48PMMHPicucsSQogiSwKxgElN1fHRR1EsW3YCpaBKFVfmzQuidm1Pc5cmhBBFmgRiAbNr1zWWLj2BpaWGkJBajBhRFxsbS3OXJYQQRZ4EYgGg1yssLDQAtG1bjiFDatGuXQVq1ZJeoRBCmIpMqjGz/ftjaNXqB06ejDcsmzChgYShEEKYmASimaSkZDJ58l5ee+0Xzp1LZMmSP81dkhBCFGsyZGoGe/deZ/ToSC5fTkKr1fDuu3UYOrS2ucsSQohiTQLRhO7dy2TGjIN8+eUpAKpXd2P+/CBq1vQwc2VCCCEkEE0oISGNtWvPodVqGDasDkOG1MbaWmaQCiFEQSCBmM/u3cvEzk6LhYWGMmUcmT8/iHLlnKhZ093cpQkhhPgHmVSTj3bvvkbr1j+wevVpw7L27StIGAohRAEkgZgPkpMzGD9+N927/8qVK8n8/HM0SilzlyWEECIXMmSaxyIjrzF6dCTXriVjZWXBiBF1CQmphUajMXdpQgghciGBmEdSU3VMmbKPb745A8Dzz3swf34QVau6mbkyIYQQxpBAzCNWVhYcPx6HtbUFI0fW4513nkerlRFpIYQoLCQQn8HduxnodHrc3GzRai1YtKgFer2iShXpFQohRGEjgfiUtm+/wpgxu6hd25OlS4PRaDRUruxq7rKEMAulFGlpaej1+ic+Xh4bG0t6eno+VVZ0SDvlpJTCwsICW1vbPJujIYH4hO7cSWfq1P2sWXMWgJIl7UlOzsTR0drMlQlhPmlpaVhZWaHVPvlXilarxdJSLlDxONJOD9LpdKSlpWFnZ5cn25NAfAJbt/7N2LG7uXHjHjY2loweXY8BA56TY4Wi2NPr9U8VhkI8C61Wm6e9ZvkEG0EpxZgxu/juu78AqFPHi/nzA2WIVIj/J6cVCXPJy8+edG2MoNFocHS0xsbGksmTG7JhQwcJQyEKmDJlytCmTRtatWpFr169uHPnjuG5v/76i65du9K8eXOaNm3K/Pnzc1wsY9u2bbz00ku0aNGCtm3b8sEHH5jjJeTq+PHjjBo1ytxlPNL+/ft54YUXKFu2LL/88ssj1/vzzz9p3bo1TZs2ZfLkyYb34fbt27z++us0bdqU119/ncTERAC2bNnCxx9/bIqXIIH4KImJ6Zw48b+b9o4dG8Dvv3dm0KDnsbSUZhOioLG1tWXLli1s27YNFxcXvvzySwBSU1Pp06cPQ4YMYdeuXURERHD48GFWrVoFwJkzZ5g0aRKffPIJO3bsYNOmTZQvXz5Pa9PpdM+8jYULF9K3b1+T7vNJlC5dmvnz5/Pqq6/mut6ECROYPXs2u3fv5uLFi2zfvh2AxYsX06xZM/bs2UOzZs1YvHgxAMHBwWzZsoXU1NT8fgkSiA/z+++XadXqB/r02UxSUgYAdnZaKlVyMW9hQgij1KtXjxs3bgCwfv16AgICCAoKAsDOzo5p06YRFhYGwKeffsrQoUOpVKkSAJaWlvTq1euBbd67d48RI0bQunVrgoOD2bhxIwCVK1c2rPPLL78wfPhwAIYPH864ceN4+eWXmTZtGg0bNszRa23atCm3bt0iPj6e/v37065dO9q1a0dUVNQD+05OTubUqVPUqFEDgKNHj9KhQwfatm1Lx44dOX/+PADh4eH07t2brl270r17d1JSUhg5ciTt27enbdu2bN68GYArV67QqVMnXnjhBV544YWH7vNJlSlThurVq2Nh8ehYiY2NJSkpiXr16qHRaOjSpQu//fYbAJs3b6Zr164AdO3a1bBco9HQuHFjtmzZ8sw1Po7JjiEeO3aMlStXotfrad269QP/RWRmZhIWFsaFCxdwdHRk+PDheHl5mao8AG7fTmPKlH38+GP2h6t+/ZLcvZshM0iFeAI+35XOl+1ef+OaUetlZWWxe/du3njjDSB7uPT555/PsU758uVJSUkhKSmJv/76i4EDBz52uwsWLMDR0ZGtW7cCGIb0chMTE8OGDRuwtLREr9fz22+/0b17d44cOYKvry+enp4MHjyY/v3706BBA65du8abb77Jzp07c2znjz/+oGrVqobHlSpVYt26dWi1WiIjI/noo49YunQpkD20GhERgaurKzNnzqRp06bMmzePO3fu0L59e5o3b46Hhwffffcdtra2XLhwgcGDB7Np06YH6u/UqRPJyckPLJ88eTKBgYGPff3/duPGDUqVKmV4XKpUKcM/LnFxcZQsWRIALy8v4uLiDOvVqlWLgwcP0rFjxyfe55MwSSDq9XqWL1/OpEmTcHd3Z8KECQQEBODr62tYZ9u2bTg4OPDJJ5+wZ88evvnmG0aMGGGK8gD46WQV+s/+gZs3U7G1tWT8+Pr07VtDhkeFKCTS0tJo06YNN27coHLlyk/1hZ2bXbt28emnnxoeu7i4PPZ3Xn75ZcOpEh06dGDBggV0796dDRs2GL7cd+3axdmzZw2/k5yczL1793BwcDAsu3nzJu7u/7tLzt27dxk+fDgXL15Eo9GQmZlpeC4wMBBX1+w5DpGRkWzZsoUlS5YAkJ6ezrVr1yhZsiTvvfcep06dwsLCggsXLjy0/nXr1j32NeYHjUaTY7KMh4cHsbGx+b5fkwTi+fPn8fb2NqR/kyZNiIqKyhGIhw4dMnSXGzVqxIoVK1BKmWT22qif2jIvsgmQSoMGJZk7N4iKFZ3zfb9CFEXG9uTu02q1eXK86/4xxNTUVN58802+/PJL+vXrh7+/P/v378+x7uXLl7G3t8fR0RF/f3+OHz9uGI58Uv/8jvr3KQD29vaGnwMCArh06RLx8fFs3ryZYcOGAdkdhp9//hlbW9tcX9s/t/3xxx/TpEkTli9fzpUrV+jSpctD96mU4osvvjAMB983d+5cPD092bJlC3q9nooVKz50v3ndQ/T29iYmJsbwOCYmBm9vb+B/oVeyZEliY2Nz/AOQlpaWa/vkFZMEYkJCQo4X5+7uzrlz5x65jqWlJfb29iQlJeHk5JRjvYiICCIiIgCYNWsWHh4ez1xfq0oXWbIvgA9ntSMkJAALC5lC/iharTZP2ryoK27tFBsb+0znIebVOYxarRZHR0dmzJhB79696devH127djWMPAUFBZGamsr777/PkCFD0Gq1DBkyhL59+9K4cWP8/PzQ6/V89dVXDxxHDAoKYvXq1UybNg3IHjJ1cXHB09OTCxcuUKlSJTZv3kyJEiXQarVYWFhgaWmZ47W1a9eOqVOn4u/vbzgk1KJFC1atWsXgwYMBOHHiBDVr1syx76pVq/LFF18YtpWcnEzp0qXRarX88MMPaDQaw4n7FhYWhvVatmzJqlWrmDFjBhqNhuPHj/Pcc8+RnJyMj48P1tbWfPfdd2RlZT30Pfj555+f+D142Ou+r3Tp0jg6OnLs2DHq1avHjz/+SL9+/dBqtbz44ov8+OOPDB06lB9//JGXXnrJsI3Lly9TrVq1h27TxsYmz/7WCt15iMHBwQQHBxse/3Oc+WnV+XAHf31gh4VFKgkJ8Y9dvzjz8PDIkzYv6opbO6Wnpz/1VVTyqocI/5tZWa1aNapWrcoPP/xAly5dWLFiBZMnT2b8+PHo9Xpee+01evXqhU6no0qVKoSGhjJw4EBSU1PRaDQEBwc/UNPQoUOZOHEigYGBWFhYMHLkSNq1a8eECRPo2bMnbm5u1KpVi3v37qHT6dDr9WRlZeXYzssvv0y7du2YP3++YfnUqVOZOHEiLVq0QKfT0bBhQz766KMc+65QoQJ3794lMTGREiVKMGjQIIYPH868efNo3bo1Sil0Oh1ZWVno9XrDtocOHcqUKVNo0aIFer2eMmXKsHr1at566y0GDBhAeHg4LVu2xN7e/pnfg2PHjtGvXz/u3LnD5s2bmT17tmEGaZs2bQyTYmbMmMGIESNIS0ujZcuWBAUFodPpeOeddxg0aBDffPMNvr6+LFmyxFDTrl27mDBhwkNrTE9Pf+BvzcfH56leg0aZ4M61Z8+eZe3atbz33nvA/8alO3XqZFhn+vTpdO3aFX9/f7KyshgwYADLli177JDp9evX86TG4vYF9rSknYxT3NopJSUlx1Ddk8jLQCzKli1bhr29PW+++aa5SzGpW7duMXjwYL7//vuHPv+wz97TBqJJZoz4+fkRExPDzZs30el07N27l4CAgBzr1KtXjx07dgDZJ3jWqFFDrn4hhBD/r3fv3lhbF78Z79euXeP99983yb5M0kMEOHLkCKtWrUKv19OyZUs6d+5MeHg4fn5+BAQEkJGRQVhYGBcvXqREiRIMHz7cMAknN9JDNC1pJ+MUt3aSHmL+k3Z6uLzsIZosEPOLBKJpSTsZp7i1kwRi/pN2erhCN2QqhCjaCvn/1aIQy8vPngSiEOKZWVhYSO9FmJxOp8v1UnFPqtCddiGEKHhsbW1JS0sjPT39iSfD2djYyJ3gjSDtlJNSCgsLizw9YV8CUQjxzDQazVPftby4HW99WtJO+U+GTIUQQggkEIUQQghAAlEIIYQAisB5iEIIIURekB7i/xs/fry5SygUpJ2MI+1kPGkr40g7Ge9p20oCUQghhEACUQghhAAkEA3+eY9F8WjSTsaRdjKetJVxpJ2M97RtJZNqhBBCCKSHKIQQQgDF7NJtx44dY+XKlej1elq3bs2rr76a4/nMzEzCwsK4cOECjo6ODB8+HC8vL/MUa2aPa6tffvmFrVu3YmlpiZOTE++88w6enp7mKdaMHtdO9+3fv5958+Yxc+ZM/Pz8TFtkAWFMW+3du5e1a9ei0WgoV64cw4YNM32hZva4doqLi2Px4sXcu3cPvV7Pm2++Sd26dc1TrBl9+umnHDlyBGdnZ+bOnfvA80opVq5cydGjR7GxsSEkJISKFSvmvlFVTGRlZakhQ4aoGzduqMzMTDV69Gh15cqVHOv89ttv6vPPP1dKKbV79241b948c5Rqdsa01fHjx1VaWppSSqnNmzcXy7Yypp2UUiolJUW9//77auLEier8+fNmqNT8jGmr69evqzFjxqikpCSllFKJiYnmKNWsjGmnJUuWqM2bNyullLpy5YoKCQkxR6lmd/LkSRUdHa1Gjhz50OcPHz6spk+frvR6vfrrr7/UhAkTHrvNYjNkev78eby9vSlZsiRarZYmTZoQFRWVY51Dhw7RokULABo1asSJEyeK5X3ejGmrmjVrYmNjA0DlypVJSEgwR6lmZUw7AYSHh/PKK69gZWVlhioLBmPaauvWrbzwwguUKFECAGdnZ3OUalbGtJNGoyElJQXIvjmuq6urOUo1u+rVqxs+Kw9z6NAhAgMD0Wg0+Pv7c+/ePW7fvp3rNotNICYkJODu7m547O7u/sCX+D/XsbS0xN7enqSkJJPWWRAY01b/tG3bNmrXrm2CygoWY9rpwoULxMXFFcshrX8ypq2uX79OTEwMkydP5r333uPYsWMmrtL8jGmnrl27smvXLgYNGsTMmTPp27evqcssFBISEvDw8DA8ftz3GBSjQBT5IzIykgsXLtCxY0dzl1Lg6PV6Vq9ezdtvv23uUgoFvV5PTEwMU6ZMYdiwYXz++efcu3fP3GUVOHv27KFFixYsWbKECRMm8Mknn6DX681dVpFQbALRzc2N+Ph4w+P4+Hjc3NweuU5WVhYpKSk4OjqatM6CwJi2Avjzzz9Zt24dY8eOLZbDgY9rp7S0NK5cucIHH3zA4MGDOXfuHLNnzyY6Otoc5ZqVsX9/AQEBaLVavLy8KFWqFDExMaYu1ayMaadt27bRuHFjAPz9/cnMzCyWI1mP4+bmluP+kY/6HvunYhOIfn5+xMTEcPPmTXQ6HXv37iUgICDHOvXq1WPHjh1A9qzAGjVqPPHdv4sCY9rq4sWLLF26lLFjxxbLYz3w+Hayt7dn+fLlLF68mMWLF1O5cmXGjh1bLGeZGvOZatCgASdPngTg7t27xMTEULJkSXOUazbGtJOHhwcnTpwA4OrVq2RmZuLk5GSOcgu0gIAAIiMjUUpx9uxZ7O3tH3u8tVidmH/kyBFWrVqFXq+nZcuWdO7cmfDwcPz8/AgICCAjI4OwsDAuXrxIiRIlGD58eLH7g7zvcW314Ycf8vfff+Pi4gJk/5GOGzfOvEWbwePa6Z9CQ0N56623imUgwuPbSinF6tWrOXbsGBYWFnTu3JmmTZuau2yTe1w7Xb16lc8//5y0tDQAevbsSa1atcxctektWLCAU6dOkZSUhLOzM926dUOn0wHQtm1blFIsX76cP/74A2tra0JCQh77t1esAlEIIYR4lGIzZCqEEELkRgJRCCGEQAJRCCGEACQQhRBCCEACUQghhAAkEIXIVWhoKFu3bjV3GbnatWsX06ZNe+Tzp0+fLpZ3jRDiSclpF6LYGDx4MImJiVhY/O//wIULF+Z69YrQ0FCaN29O69at86yO0NBQzp07h4WFBdbW1lSrVo1+/frl2UWau3XrxqJFi/D29s6T7T3K999/z7p169BqtVhaWuLr68vbb7+Nv79/gapTCGMVq/shCjFu3Dief/55c5dB3759ad26NcnJycydO5dVq1YxfPhwc5f1xBo3bszQoUPJysri+++/Z968eSxZssTcZQnxVCQQRbGWnJxMWFgY586dQ6/XU6VKFfr375/jjgP33bhxg88++4xLly6h1WqpWbMmI0aMAODatWusWLGCCxcu4OTkRPfu3WnSpMlj91+iRAkaNmzIli1bAPjrr7/48ssvuX79Oj4+PvTu3ZsqVaoAsGPHDn744Qfu3r2Lo6Mjr7/+Os2bN2fHjh1s3bqVDz/8kClTpgAwZswYAN555x2cnZ355JNPWLJkCevXryc6OppRo0YZali5ciVKKfr27UtKSgqrVq3i6NGjaDQaWrZsSbdu3XL0qh/G0tKS5s2bs27dOu7evYuTkxPnz59n5cqVXLt2DWtraxo2bEivXr3QarUPrbNJkyYcPnyYNWvWcOvWLXx9fenfvz/lypV7bDsKkRckEEWxppSiRYsWjBgxAr1ez2effcby5csZO3bsA+uuWbOGWrVqMWXKFHQ6HRcuXACyL+I9bdo0unXrxsSJE/n777+ZNm0aZcuWxdfXN9f93717lwMHDlC+fHmSk5OZNWsWffr0oWnTpuzbt49Zs2axaNEirKysWLlyJTNnzsTHx4fbt2+TnJz8wPY++OADunXrxscff2wYirx/fVCApk2b8sMPP5CamoqdnR16vZ59+/YxevRoABYvXoyzszOLFi0iPT2dWbNm4e7uTps2bXJ9HTqdjp07d+Lo6IiDgwMAFhYW9OrVCz8/P+Lj45k5cyabN2+mffv2D63z4sWLfPbZZ4wbNw4/Pz8iIyOZPXs2CxYsKJYXjxemJ5NqRLHy8ccf07t3b3r37s3s2bNxdHSkUaNG2NjYYGdnR+fOnTl9+vRDf1er1XLr1i1u376NtbU1VatWBbKvPenp6UnLli2xtLSkQoUKNGzYkH379j2yjpUrV9K7d2/GjBmDq6srvXr14siRI3h7exMYGIilpSXNmjXDx8eHw4cPA9k3hv3777/JyMjA1dWVMmXKPPHr9/T0pEKFChw8eBCAEydOYGNjg7+/P4mJiRw9epTevXtja2uLs7Mz7du3Z+/evY/c3r59++jduzc9evRg69atjBw5EktLSwAqVqyIv78/lpaWeHl5ERwczKlTpx65rYiICIKDg6lcuTIWFha0aNECrVbLuXPnnvh1CvE0pIcoipUxY8bkOIaYnp7OqlWrOHbsmOHee6mpqej1+geGCXv27MmaNWuYOHEiDg4OvPzyy7Rq1Ypbt25x7tw5evfubVg3KyuLwMDAR9bRp0+fBybqJCQk4OnpmWOZp6cnCQkJ2NraMnz4cH7++WeWLFlClSpVePvttylduvQTt0GzZs3Ys2cPQUFB7N6923AB7bi4OLKyshgwYIBhXaXUQ4eP77t/DPHu3bvMnTuXCxcuUKNGDSD7hr+rV68mOjqajIwMsrKyqFix4iO3FRcXx86dO/ntt98My3Q63WNv6ipEXpFAFMXazz//zPXr15kxYwYuLi5cunSJsWPH8rDJ1y4uLgwaNAiAM2fO8OGHH1K9enXc3d2pXr06kydPfqZa3NzcOHDgQI5lcXFx1K5dG4DatWtTu3ZtMjIyWLNmDZ9//jlTp0594v00btyY1atXEx8fz8GDBw2nbLi7u6PValm+fLmhl2csJycnBg4cyPjx42nWrBmurq4sW7aM8uXLM2zYMOzs7Ni4cSP79+9/5Dbc3d3p3LkznTt3fuLXJERekCFTUaylpaVhbW2Nvb09ycnJrF279pHr7tu3z3Dz1vvHyTQaDfXq1SMmJobIyEh0Oh06nY7z589z9erVJ6qlTp06xMTEsHv3brKysti7dy9Xr16lbt26JCYmEhUVRVpaGlqtFltb20feq9PZ2ZnY2NhH7sfJyYkaNWrw6aef4uXlZTjO6erqSq1atVi9ejUpKSno9Xpu3LiR6zDnP/n4+FCrVi02bNgAZPe07e3tsbW15dq1a/z++++51tm6dWu2bNnCuXPnUEqRlpbGkSNHSE1NNWr/Qjwr6SGKYq1du3YsWrSIfv364ebmxssvv0xUVNRD142OjubLL78kJSUFFxcX+vTpY7hf5qRJk1i1ahWrVq1CKUW5cuXo1avXE9Xi6OjI+PHjWblyJUuXLsXb25vx48fj5OTE7du3+eWXXwgLC0Oj0VC+fHn69+//0O107dqVxYsXk5GRwYABAx56A+dmzZoRFhZGz549cywfMmQI33zzDSNHjiQ1NZWSJUvyyiuvGP0aOnbsyNSpU+nUqRNvvfUWX3zxBRs2bKBChQo0adLEcGPbh9XZpEkTBg4cyIoVK4iJiTEcp61WrZrR+xfiWciJ+UIIIQQyZCqEEEIAEohCCCEEIIEohBBCABKIQgghBCCBKIQQQgASiEIIIQQggSiEEEIAEohCCCEEIIEohBBCAPB/lXpI3lOBtdsAAAAASUVORK5CYII=\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 绘制ROC曲线：\n",
    "from sklearn.metrics import roc_curve,auc\n",
    "from sklearn.metrics import make_scorer,f1_score\n",
    "scorer = make_scorer(f1_score,pos_label=0)\n",
    "gs = GridSearchCV(estimator=pipe_svc,param_grid=param_grid,scoring=scorer,cv=10)\n",
    "y_pred = gs.fit(X_train,y_train).decision_function(X_test)\n",
    "#y_pred = gs.predict(X_test)\n",
    "fpr,tpr,threshold = roc_curve(y_test, y_pred) ###计算真阳率和假阳率\n",
    "roc_auc = auc(fpr,tpr) ###计算auc的值\n",
    "plt.figure()\n",
    "lw = 2\n",
    "plt.figure(figsize=(7,5))\n",
    "plt.plot(fpr, tpr, color='darkorange',\n",
    "         lw=lw, label='ROC curve (area = %0.2f)' % roc_auc) ###假阳率为横坐标，真阳率为纵坐标做曲线\n",
    "plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')\n",
    "plt.xlim([-0.05, 1.0])\n",
    "plt.ylim([-0.05, 1.05])\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('Receiver operating characteristic ')\n",
    "plt.legend(loc=\"lower right\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.结语"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本章中，我们重点讨论了各种回归和分类算法的具体推导与简单应用，并且给出了如何使用sklearn这个强大的python工具库进行简单的机器学习模型的建模代码。本章的重点是各个基础算法的掌握，包括回归和分类(重点是分类)算法以及怎么用网格搜索以及其他搜索方式进行调参。简单模型在进行复杂项目的时候往往显得力不从心，那么在下一章中，我们将开始本次开源项目的主题----集成学习，我们着重讨论如何将本章所学的基础模型进行集成，变成功能更加强大的集成模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了巩固本章的理解，在这里给个小任务，大家结合sklearn的fetch_lfw_people数据集，进行一次实战。fetch_lfw_people数据集是一个图像数据集，详细内容可以参照：                   \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_lfw_people.html                   \n",
    "案例的内容是对图像进行识别并分类。               \n",
    "参考资料：                \n",
    "https://blog.csdn.net/cwlseu/article/details/52356665                      \n",
    "https://blog.csdn.net/jasonzhoujx/article/details/81905923"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}